oversight

Arkansas' Adult Education and Family Literacy Act Program

Published by the Department of Education, Office of Inspector General on 2010-05-28.

Below is a raw (and likely hideous) rendition of the original report. (PDF)

       Arkansas’ Adult Education and Family Literacy Act
                          Program

                                 FINAL AUDIT REPORT




                                    ED-OIG/A06J0001
                                       May 2010


Our mission is to promote the                         U.S Department of Education
efficiency, effectiveness, and                        Office of Inspector General
integrity of the Department's
programs and operations.




 
                                     NOTICE

       Statements that managerial practices need improvements, as well as other
     conclusions and recommendations in this report, represent the opinions of the
    Office of Inspector General. Determinations of corrective action to be taken will
             be made by the appropriate Department of Education officials.

   In accordance with the Freedom of Information Act (5 U.S.C. § 552), reports
issued by the Office of Inspector General are available to members of the press and
     general public to the extent information contained therein is not subject to
                               exemptions in the Act.




 
                            UNITED STATES DEPARTMENT OF EDUCATION
                                 OFFICE OF INSPECTOR GENERAL

                                                                                 AUDIT SERVICES
                                                                                 Dallas Audit Region

                                          May 28, 2010


William L. Walker, Jr.
Director
Arkansas Department of Career Education
Luther Hardin Building
Three Capitol Mall
Little Rock, AR 72201

Dear Mr. Walker:

Enclosed is our final audit report, Control Number ED-OIG/A06J0001, entitled Arkansas'
Adult Education and Family Literacy Act Program. This report incorporates the comments you
provided in response to the draft report. If you have any additional comments or information
that you believe may have a bearing on the resolution of this audit, you should send them directly
to the following Department of Education official, who will consider them before taking final
Departmental action on this audit:

                              Brenda Dann-Messier
                              Assistant Secretary
                              Office of Vocational and Adult Education
                              U.S. Department of Education
                              400 Maryland Avenue, S.W.
                              Washington, DC 20202-7100

It is the policy of the U. S. Department of Education to expedite the resolution of audits by
initiating timely action on the findings and recommendations contained therein. Therefore,
receipt of your comments within 30 days would be appreciated.

In accordance with the Freedom of Information Act (5 U.S.C. § 552), reports issued by the
Office of Inspector General are available to members of the press and general public to the extent
information contained therein is not subject to exemptions in the Act.


                                             Sincerely,

                                             /s/
                                             Gary D. Whitman
                                             Regional Inspector General for Audit

Enclosures

 
                                              TABLE OF CONTENTS

                                                                                                                                 Page

EXECUTIVE SUMMARY ...........................................................................................................3 

BACKGROUND ............................................................................................................................5 

AUDIT RESULTS .........................................................................................................................7 

          FINDING NO. 1 – Arkansas Did Not Adequately Monitor Provider
                          Performance ......................................................................................7 

          FINDING NO. 2 – Arkansas Awarded New Grants to Providers That
                          Had Not Demonstrated Their Effectiveness ...................................9 

          FINDING NO. 3 – Arkansas and Local Providers Did Not Use Adult
                          Education Grant Funds in Compliance with Fiscal
                          Requirements...................................................................................15 

OTHER MATTERS ....................................................................................................................18 

OBJECTIVES, SCOPE, AND METHODOLOGY ..................................................................19 

Enclosure 1: Selected Excerpts from the Applicable Law and OMB Circular A-87 ............21 

Enclosure 2: Auditee Comments ...............................................................................................31 

Enclosure 3: Arkansas E&E Calculation .................................................................................42 




 
             Acronyms/Abbreviations Used in this Report


AEFLA           Adult Education and Family Literacy Act

AED             Arkansas Department of Career Education’s Adult Education Division

Arkansas        Arkansas Department of Career Education

AERIS           Adult Education Reporting Information System

C.F.R.          Code of Federal Regulations

DAEL            United States Department of Education, Office of Vocational and Adult
                Education, Division of Adult Education and Literacy

Department      United States Department of Education

E&E             Effective and Efficient

EFL             Educational Functioning Levels

E/L Civics      English Literacy and Civics Education

GED®            General Educational Development

MIS             Management Information System

NRS             National Reporting System for Adult Education

OMB             Office of Management and Budget

OVAE            United States Department of Education, Office of Vocational and Adult
                Education

RFP             Request for Proposal

U.S.C.          United States Code

GSA             United States General Services Administration

WIA             Workforce Investment Act of 1998




 
Final Report
ED-OIG/A06J0001                                                                                    Page 3 of 43



                                                               EXECUTIVE SUMMARY


The objectives of our audit were to determine whether (1) the Arkansas Department of Career
Education (Arkansas) adequately monitored the performance of providers receiving adult
education funds; (2) Arkansas and its providers used adult education funds in accordance with
law, regulations, and guidance; and (3) Arkansas awarded adult education grants to providers
appropriately given the considerations identified in Section 231 of the Adult Education and
Family Literacy Act (AEFLA).1 Our audit covered the period July 1, 2007, through June 30,
2008 (program year 2007-2008). We found that:

       1. Arkansas did not adequately monitor the performance of providers receiving adult
          education funds. As a result, Arkansas and the U.S. Department of Education
          (Department), Office of Vocational and Adult Education (OVAE), had no assurances that
          the local providers were meeting the requirements of the grant and serving adults to allow
          them to increase their literacy and basic education (Finding No. 1).

       2. Arkansas awarded seven literacy councils new grants as local providers for program year
          2008-2009 even though they did not meet Arkansas’ definition of effectiveness during
          the prior grant period. According to the AEFLA, to be considered an eligible provider, a
          literacy council’s program must show demonstrated effectiveness (Finding No. 2).

       3. Arkansas did not ensure that $521,259 in adult education funds were expended in
          accordance with the AEFLA, regulations, and guidance (Finding No. 3).

       4. For program year 2003-2004, Arkansas did not award adult education grants in
          accordance with the AEFLA. However, OVAE has addressed the issue, and we have
          included the issue in this report only as information.

We recommend that the Assistant Secretary, OVAE, require Arkansas to—

             Enhance its monitoring process so that it (a) has assurances that providers met the
              required benchmarks before being awarded new grants, and (b) takes appropriate actions
              if providers do not meet the required benchmarks;

             Provide documentation that shows the seven literacy councils in our sample met
              Arkansas’ definition of effectiveness during the prior grant period or return to the
              Department $62,144 that was improperly awarded to those literacy councils for program
              year 2008-2009;




                                                            
1
    The AEFLA is Title II of the Workforce Investment Act (WIA) of 1998, Public Law 105-220, and is codified at
    20 U.S.C. § 9201 et.seq.
 
Final Report
ED-OIG/A06J0001                                                                       Page 4 of 43

       Identify all other literacy councils who received new awards for program year 2008-2009
        and determine whether they met Arkansas’ definition of effectiveness during the prior
        grant period;

       Provide OVAE with assurances that all literacy councils and community-based
        organizations that were awarded 2008-2009 grants were effective, and, therefore, met the
        provider eligibility requirements of the AEFLA;

       If any of the literacy councils were not effective, return the improperly awarded AEFLA
        funds to the Department;

       Return to the Department $13,027 in unallowable costs; and

       Provide adequate documentation to support $508,232 in inadequately documented costs
        or return the inadequately documented amount to the Department.

We provided a draft of this report to Arkansas on October 5, 2009. In its comments on the draft
report, Arkansas did not agree with Finding Nos. 1 and 2 or their recommendations and agreed
with only the portion of Finding No. 3 concerning payments made to General Educational
Development (GED®) testers being unallowable and agreed to return the funds. Arkansas’
comments are summarized at the end of each finding. The full text of Arkansas’ comments is
included as Enclosure 2 in this report. Because of the voluminous amount of documentation
provided, we did not include all the additional documentation that Arkansas submitted with its
comments; however, the documentation will be made available upon request.

We considered Arkansas’ comments on Finding Nos. 1 and 2 and again reviewed the
documentation referenced in the comments. Based on our additional review, we did not change
Finding No. 1 and its corresponding recommendation. For Finding No. 2, Arkansas provided
evidence showing that it defined effectiveness for the AEFLA program. Therefore, we adjusted
the finding to explain that, even with the established criteria, Arkansas did not ensure that
literacy councils met the eligible provider criteria prior to awarding new grants for program year
2008-2009. We made some technical changes to the report for clarity. We also deleted
recommendations related to developing a definition of effectiveness. Further, we added a
recommendation related to Arkansas evaluating the effectiveness of literacy councils and
providing the Department with documentation and assurances that demonstrate all literacy
councils that received program year 2008-2009 awards met the definition of effectiveness for the
prior grant period.

For Finding No. 3, we reviewed Arkansas’ comments and the additional documentation
provided. We concluded that Arkansas provided sufficient documentation to support the
allowability of the majority of the inadequately documented supply purchases. Therefore, we
adjusted the dollar amounts in the final report. We also reviewed the self-certifications of time
that Arkansas submitted for review. However, we did not make changes to the finding
concerning inadequately documented payroll expenses, because the additional documentation
was at least a year after the fact and did not satisfy the requirements set forth in Office of
Management and Budget (OMB) Circular A-87.


 
Final Report
ED-OIG/A06J0001                                                                                     Page 5 of 43




                                                               BACKGROUND


The AEFLA provides grants to States to fund local programs of adult education and literacy
services, including workplace literacy services, family literacy services, and English literacy and
civics education programs. Participation in these programs is limited to adults and out-of school
youths age 16 and older. Adult education funds are distributed to States using a formula
established by Congress. The funds are distributed to local providers for adult education and
literacy services. States must match 25 percent of the Federal contribution with State or local
funds.

States are required to competitively award 82.5 percent of their Federal grants to local school
districts, community colleges, community- and faith-based organizations, and other providers to
support adult education programs. States are allowed to retain 17.5 percent of the Federal
allocation for program improvement activities (12.5 percent), such as professional development
for instructors, and administrative expenses (5 percent).

Arkansas awards three types of adult education grants: Direct and Equitable, English Literacy
and Civics Education (E/L Civics), and Correctional. Providers can apply for and receive more
than one type of grant in a program year. For program year 2007-2008, Arkansas received
$5,609,279 in AEFLA funding. The initial adult education grants were awarded for program
year 2003-2004, with continuation grants awarded annually through program year 2007-2008.

The National Reporting System for Adult Education (NRS) is an outcome-based reporting
system for the State-administered, federally funded adult education program. Developed by the
Department’s Division of Adult Education and Literacy (DAEL), the NRS continued a
cooperative process through which State adult education directors and DAEL manage a reporting
system that demonstrates learner outcomes for adult education. After passage of the WIA, NRS
became mandatory.

Benchmarks

The AEFLA requires programs to develop performance measures that are in compliance with
Section 212(b)(2)(A)(i-iii). On an annual basis, Arkansas negotiated with OVAE 15 benchmarks
for the adult education grant. Arkansas negotiated with OVAE for 11 educational outcome
benchmarks plus 4 follow-up outcome benchmarks.2 According to State policy, local providers
are required to meet 80 percent of the negotiated benchmarks annually to be considered eligible
for funding. The established benchmarks for program years 2003-2004 through 2007-2008 are
provided in Table 1. The percentages in the table show the percentage of enrolled students who
passed and could have advanced to the next level.




                                                            
2
    For program year 2003-2004, there was one additional educational outcome benchmark (1.6 in Table 1).
 
Final Report
ED-OIG/A06J0001                                                                                                  Page 6 of 43


        Table 1. Statewide Negotiated Benchmarks from 2003-2004 through 2007-2008
    Indicators                                   2003-2004       2004-2005       2005-2006       2006-2007        2007-2008

    1.1 ABE Beginning Literacy                       26%             45%             49%             53%             43%
    1.2 ABE Beginning Basic Education                29%             46%             51%             49%             42%
    1.3 ABE Intermediate Low                         33%             42%             48%             50%             46%
    1.4 ABE Intermediate High                        33%             44%             49%             50%             47%
    1.5 ASE Low                                      28%             44%             51%             53%             44%
    1.6 ASE High                                    32%              N/A             N/A             N/A             N/A
    1.7 ESL Beginning Literacy                       25%             39%             46%             51%             37%
    1.8 ESL Low Beginning Literacy*                  25%             34%             44%             47%             36%
    1.9 ESL High Beginning*                          30%             36%             47%             47%             36%
    1.10 ESL Low Intermediate*                       30%             33%             44%             54%             45%
    1.11 ESL High Intermediate*                      30%             36%             40%             46%             45%
    1.12 ESL Advanced*                               30%             24%             31%             49%             47%
    Entered Employment                               35%             60%             70%             72%             58%
    Retained Employment                              45%             64%             71%             86%             62%
    Obtained a GED®                                 46%              70%             73%             77%             60%
    Entered Post Secondary Ed.                       35%             60%             68%             73%             38%
    * Starting with program year 2006-2007, the title of the indicators changed. The names of the indicators in the table
    reflect the change.
    ABE means Adult Basic Education.
    ASE means Adult Secondary Education.
    ESL means English as a Second Language.


Effective and Efficient (E&E) Calculation

Arkansas developed an E&E calculation to assist in evaluating each local provider’s
effectiveness. (See Enclosure 3). Beginning with program year 2005-2006, Arkansas used the
E&E calculation to compute an overall benchmark score for each local provider. Arkansas
tested this method during program year 2006-2007 and implemented it during program year
2007-2008. Arkansas considered program year 2006-2007 to be a transition year during which
providers would be held harmless. Arkansas’ Adult Education Division (AED) tracked each
provider’s E&E for 2006-2007 but did not plan to enforce State policy until program year 2007-
2008.




 
Final Report
ED-OIG/A06J0001                                                                     Page 7 of 43




                                    AUDIT RESULTS


As of program year 2007-2008, Arkansas had not adequately monitored providers as required by
the AEFLA, resulting in potentially ineligible literacy councils receiving grant awards as local
providers for program year 2008-2009. Based on our review of benchmarks, the literacy
councils had not demonstrated effectiveness, a requirement for a literacy council to be eligible
for funding. Also, Arkansas did not always ensure that adult education funds were being
expended in accordance with the law, regulations, and guidance.


FINDING NO. 1 – Arkansas Did Not Adequately Monitor Provider
                Performance
As of program year 2007-2008, Arkansas had not adequately monitored the performance of adult
education providers. To continue to receive Federal funds, local providers must show that they
are achieving performance goals for the adult education grant. AED has a system in place to
conduct on-site reviews and provide on-site technical assistance to local providers of adult
education. Local providers receive technical assistance visits annually, and Arkansas conducted
approximately 10 on-site program reviews every year. However, the on-site reviews did not
address program effectiveness. Beginning with program year 2005-2006, Arkansas used its E&E
calculation to evaluate the effectiveness of each local provider. However, Arkansas tested this
method only during program year 2006-2007 and did not plan to enforce State policy until
program year 2007-2008.

To test whether Arkansas adequately monitored the adult education providers, we reviewed the
reports for the 10 monitoring reviews that Arkansas completed during program year 2007-2008.
For the 10 reviews completed between June 2007 and December 2007, four corrective action
plans were not in place as of April 2009. During our audit, we were not provided any
performance improvement plans submitted by the provider to the State.

According to 34 Code of Federal Regulations (C.F.R.) § 80.40,“[g]rantees must monitor grant
and subgrant supported activities to assure compliance with applicable Federal requirements and
that performance goals are being achieved. Grantee monitoring must cover each program,
function or activity.”

Because Arkansas did not adequately monitor the performance of its local adult education
providers, both Arkansas and OVAE had no assurances that the local providers were meeting the
requirements of the grant and serving adults to allow them to increase their literacy and basic
education. We randomly selected 25 local providers from the universe of 76 local providers who
received adult education funds for program year 2007-2008. We reviewed NRS data from
program year 2003-2004 through program year 2007-2008 and compared local provider
performance information to the benchmarks Arkansas negotiated with OVAE. None of the 25
local providers that we reviewed met at least 80 percent of the benchmarks every year, and 7 of 8

 
Final Report
ED-OIG/A06J0001                                                                      Page 8 of 43

literacy councils in our sample did not meet at least 80 percent of the benchmarks for 3
consecutive years (See Finding No. 2).

Recommendations

We recommend that the Assistant Secretary, OVAE, require Arkansas to—

1.1   Enhance the monitoring process so that it (a) has assurances that providers met the required
      benchmarks before being awarded new grants, and (b) takes appropriate actions if
      providers do not meet the required benchmarks.

Auditee Comments

Arkansas did not agree with our finding that it did not adequately monitor the performance of
local providers. In its comments on the draft report, Arkansas stated—

       AED monitors all providers to assure compliance with applicable federal
       requirements and assures that performance goals are being achieved. AED
       reviews each core indicator of performance for each local provider with a robust
       evaluation calculation. AED has a state E&E policy, approved by OVAE, which
       implements a process of corrective action for providers that are not meeting the
       required benchmarks.

Arkansas continued by outlining the history of monitoring its adult education program. Arkansas
stated that, prior to 2000, quality indicators of performance were used for program reviews and
annual E&E reports were completed for each local provider’s program. In 2000, the NRS
became the system of accountability in adult education. Arkansas, in its State Unified Plan for
the period 2000-2005, agreed to evaluate the effectiveness of adult education and literacy
activities based on the core indicators of performance with the data provided through a State
management information system (MIS). The State plan further indicated new MIS software was
being developed for implementation in July 2000. However, Arkansas struggled with three
different unsuccessful systems between 2001 and 2005, and AED was forced to rely on local
programs keeping their own student data records in the programs’ own student management
systems. The local programs then reported student data to the State, which in turn aggregated the
data and reported to OVAE via the NRS.

Arkansas stated that, when the Federal review team from the DAEL visited Arkansas in April
2006, one of its recommendations was that Arkansas strengthen the local onsite monitoring
process. However, DAEL did not cite the issue as a finding. In response to DAEL’s
recommendation, AED adhered to the corrective action plan and adopted a process that included
timelines and closure procedures that are essential to continuous program improvement at the
local level.




 
Final Report
ED-OIG/A06J0001                                                                         Page 9 of 43

OIG Response

The comments did not change our determination that, for program year 2007-2008, Arkansas did
not adequately monitor local providers’ performance. Therefore, we did not modify our
recommendation that Arkansas enhance its monitoring process.

Arkansas stated that it made changes to its monitoring process to correct weaknesses in the
monitoring of local programs. During the period 2000 through 2005, it struggled with
implementing an adequate management information system for capturing data on local provider
performance. We acknowledge that Arkansas encountered difficulties in creating and
implementing a successful State system of data management from 2000 through 2005. The
result of these difficulties, though, meant that Arkansas did not have the performance data
necessary to take timely and appropriate action against nonperforming providers. Despite these
difficulties, Arkansas had a regulatory responsibility to monitor provider performance to ensure
that performance goals were being achieved.

We were aware that Arkansas’ monitoring process was reviewed by DAEL in April 2006, and
that DAEL recommended that the monitoring of the local programs should be strengthened but
did not cite the issue as a “finding.” Nevertheless, Arkansas agreed to strengthen the monitoring
process. Although Arkansas stated that it had implemented an adequate onsite monitoring
process, our review of Arkansas’ policy and procedures and tests of the process showed that the
process was not fully implemented for program year 2007-2008. Therefore, we cited the issue as
a finding.


FINDING NO. 2 – Arkansas Awarded New Grants to Providers That Had Not
                Demonstrated Their Effectiveness
For the program year 2008-2009 grant competition, Arkansas awarded adult education grants to
providers that had not demonstrated that they were effective adult education providers during the
previous 5-year grant period. According to the AEFLA, to be considered an eligible provider, a
literacy council’s program must show demonstrated effectiveness. The WIA, Title II Section
203 (5), defines an “Eligible Provider” as one of the following:

       (A)   a local educational agency;
       (B)   a community-based organization of demonstrated effectiveness;
       (C)   a volunteer literacy organization of demonstrated effectiveness;
       (D)   an institution of higher education;
       (E)   a public or private nonprofit agency;
       (F)   a library;
       (G)   a public housing authority;
       (H)   a nonprofit institution that is not described in any of the
             subparagraphs (A) through (G) and has the ability to provide literacy
             services to adults and families; and,
       (I)   a consortium of the agencies, organizations, institutions, libraries, or
             authorities described in any of subparagraphs (A) through (H).


 
Final Report
ED-OIG/A06J0001                                                                                     Page 10 of 43

According to State policy, local providers are required to meet or exceed 80 percent of
negotiated benchmarks to be considered effective, efficient, and eligible for funding. Local
providers not meeting 80 percent of the negotiated core indicators of performance in the first
year are to submit an improvement plan. The improvement plan is to address each core indicator
of performance that was not met and include measurable objectives for correcting deficiencies
during the following year. Local providers that do not meet at least 80 percent of negotiated
benchmarks for a second consecutive year are to be assigned a State technical assistance team to
assist the provider in addressing its deficiencies. Local providers not achieving 80 percent of
benchmarks for 3 consecutive years are to be submitted to the State Board of Workforce
Education and Career Opportunities for appropriate action prior to continuation of funding.

To assess whether Arkansas should have considered local providers effective, we obtained
information from NRS and compared provider performance data against the negotiated
benchmark percentages for 25 randomly selected providers from the universe of 76 local
providers. Our sample included eight literacy councils that were required to demonstrate
effectiveness to be considered eligible providers and eligible for continued funding. We limited
testing to the literacy councils because only literacy councils and community-based
organizations are required to demonstrate effectiveness to meet the definition of an eligible
provider.

To assess whether the literacy councils met 80 percent of the negotiated benchmarks for their
relevant indicators,3 we calculated, using NRS data, the percentage of enrolled students who
advanced to the next level and compared the calculated percentages to Arkansas’ established
benchmarks (the benchmarks can be seen in Table 1 in the Background section of this report).
For example, for program year 2007-2008, Arkansas negotiated a benchmark of 43 percent for
indicator 1.1. The percentage is based on the number of enrolled students who advanced to the
next level for the indicator. To consider the literacy council as meeting the benchmark for the
indicator, the literacy council would have needed to advance 41 percent of the enrolled students.
The results of our comparison are shown in Table 2.




                                                            
3
  Literacy councils are not required to enroll students in all 11 indicators. They can choose the performance
indicators in which they want to enroll students. Therefore, we considered only those indicators in which students
were enrolled.
 
Final Report
ED-OIG/A06J0001                                                                                                               Page 11 of 43

             Table 2. Comparison of Literary Councils Performance with Benchmarks
                        Number of Benchmarks Attempted and Met for Each Program Year
                        2007-08      2006-07      2005-06       2004-05       2003-04
Literacy                                                                                                                      Consecutive




                                Attempted




                                                    Attempted




                                                                         Attempted




                                                                                             Attempted




                                                                                                                  Attempted
Council                                                                                                                       Years That 80
                                                                                                                              Percent of
                                                                                                                              Benchmarks
                       Met




                                             Met




                                                                 Met




                                                                                      Met




                                                                                                          Met
                                                                                                                              Were Not Met
                        1         7           3       7           0        5           3       5            5       5
Carroll                                                                                                                             4
                      (14%)                 (43%)               (0%)                 (60%)               (100%)
                                              1
                        1         7                   5            3       3           6       9            7       7
Crawford                                    (20%)                                                                                   2
                      (14%)                                     (100%)               (67%)               (100%)
                          0      3        0       3         0      5          1       4      0        5
Crittenden                                                                                                                          5
                        (0%)            (0%)              (0%)             (25%)           (0%)
                          2      9        1       9         0      9          8       8      9        9
Dogwood                                                                                                                             3
                       (22%)           (11%)              (0%)            (100%)         (100%)
                          2      7        1       6         3      4          3       5      4        5
Drew                                                                                                                                4
                       (29%)           (17%)             (75%)             (60%)          (80%)
                          3      7        3       5         2     10          9      10      9       10
Grant                                                                                                                               3
                       (43%)           (60%)             (20%)             (90%)          (90%)
                          0      4        0       5         2      6          0       5      2        3
Hot Springs                                                                                                                         5
                        (0%)            (0%)             (33%)              (0%)          (67%)
                          0      8        0       8         1      5          4       4      6        6
Monroe                                                                                                                              3
                        (0%)            (0%)             (20%)            (100%)         (100%)
Percentage by each attempted number is the percentage of benchmarks that meet the minimum standards for negotiated
benchmarks for the program year.


The results of our comparison show that seven of the eight literacy councils, for three
consecutive years or more, did not meet the negotiated benchmarks for each indicator for which
they enrolled students. These seven literacy councils did not meet Arkansas’ definition for
effectiveness and, therefore, did not meet the definition of “eligible provider” as defined in the
AEFLA, Section 203(5).

To evaluate whether a local provider is effective and efficient, Arkansas developed an E&E
calculation. However, at the time of our audit, Arkansas had not fully implemented its E&E
calculations. Even prior to the development of the E&E calculation, Arkansas was required to
monitor the performance of its providers and evaluate their effectiveness. Information from NRS
could have been used to test the effectiveness of local providers and satisfy the requirements set
forth in Section 224(b)(3) of the AEFLA. This information is and was available to Arkansas, but
Arkansas did not use the information in a timely manner to assess whether the adult education
services delivered by the literacy councils were effective. Therefore, at least seven literacy
councils received additional AEFLA grants even though they did not qualify as eligible
providers.

According to 34 C.F.R. § 80.43, there are remedies that a State may take when local provider
noncompliance is identified.

           (a)   Remedies for noncompliance. If a grantee or subgrantee materially
                 fails to comply with any term of an award, whether stated in a
                 Federal statute or regulation, an assurance, in a State plan or
                 application, a notice of award, or elsewhere, the awarding agency

 
Final Report
ED-OIG/A06J0001                                                                      Page 12 of 43

             may take one or more of the following actions, as appropriate in the
             circumstances:

             (1)   Temporarily withhold cash payments pending correction of the
                   deficiency by the grantee or subgrantee or more severe
                   enforcement action by the awarding agency,
             (2)   Disallow (that is, deny both use of funds and matching credit
                   for) all or part of the cost of the activity or action not in
                   compliance,
             (3)   Wholly or partly suspend or terminate the current award for the
                   grantee’s or subgrantee’s program,
             (4)   Withhold further awards for the program, or
             (5)   Take other remedies that may be legally available.

Arkansas did not take any of the above actions even though an analysis of NRS data showed that
seven of the eight literacy councils did not satisfy the State’s policy on effectiveness for three
consecutive years ending with program year 2007-2008. For program year 2008-2009, Arkansas
awarded $62,144 in adult education funds to the seven literacy councils in our sample that had
not met at least 80 percent of the negotiated benchmarks for at least 3 consecutive years. For the
literacy councils not in our sample, we have no assurances that the funds were awarded only to
eligible providers.

Recommendations

We recommend that the Assistant Secretary, OVAE, require Arkansas to—

2.1   Provide documentation that shows the seven literacy councils in our sample met Arkansas’
      definition of effectiveness during the prior grant period or return to the Department
      $62,144 that was improperly awarded to those literacy councils for program year
      2008-2009;

2.2   Identify all other literacy councils who received new awards for program year 2008-2009
      and

      (a) determine whether they met Arkansas’ definition of effectiveness during the prior grant
      period,
      (b) provide OVAE with assurances that all literacy councils and community-based
      organizations who were awarded 2008-2009 grants were effective, and, therefore, met the
      provider eligibility requirements of the AEFLA, and
      (c) return the improperly awarded AEFLA funds to the Department for those literacy
      councils that were not effective; and,

2.3   Establish a system of internal control adequate to provide reasonable assurance that it
      (a) follows Federal requirements for ensuring that all sub-grantees remain eligible
      providers for the entire award period, (b) monitors effectiveness of the programs that
      literacy councils and community-based organizations provide; and (c) takes appropriate,
      timely action consistent with 34 C.F.R. § 80.43 when providers materially fail to comply
      with the requirements of the award.
 
Final Report
ED-OIG/A06J0001                                                                      Page 13 of 43

Auditee Comments

Arkansas disagreed with our finding that it did not define demonstrated effectiveness for literary
councils. Arkansas stated that it has a system in place to ensure that the program effectiveness of
the literacy councils is evaluated and considered prior to awarding grant funds. Arkansas stated
that its Adult Education Reporting Information System (AERIS) was finally implemented in July
2005. AED then was able to place more confidence in the data from the local providers. In the
same year, AED introduced a model for E&E calculations, which used as its basis the core
indicators of performance benchmarks for the NRS Educational Functioning Levels (EFL) and
the four follow-up goals. In the calculation, each EFL and goal is weighted so that each EFL or
goal carries the weight of the number of students in that level or with that goal. Program year
2006-2007 was determined to be a transition year where programs would be held harmless.
AED would track each program E&E for 2006-2007, but State policy would not be enforced
until program year 2007-2008.

According to Arkansas, at the end of program year 2007-2008, 28 adult education and literacy
programs had not met the E&E benchmarks (based on the new State E&E calculation). Arkansas
conducted site reviews of these programs, and program improvement plans were put into effect.
Arkansas stated that, during program year 2008-2009, these 28 programs were monitored closely
for progress toward meeting Federal benchmarks. Based on preliminary data from 2008-2009, it
appears that only two programs will not meet the E&E benchmarks. The State integrates the
E&E policy into its in-depth program reviews by choosing to visit those programs with low E&E
results. During program year 2008-2009, the Arkansas staff made site visits to 100 percent of
the adult education and literacy programs statewide.

Arkansas stated that, prior to the new competition for program year 2008-2009, it worked
diligently with OVAE to address concerns with the last program review conducted by OVAE.
That program review required Arkansas to revamp its entire Request for Proposal (RFP) process
and application. Arkansas stated that it requested “clarification from OVAE about past
effectiveness and demonstrated effectiveness and was instructed that any volunteer organization
should have direct and equitable access to apply for the funds, whether it had a literacy program
in place or not, whether it had students or not.” Arkansas also stated that it was advised to allow
all programs to apply for funds and to make adjustments to the application for programs that did
not have NRS tables, which would allow the program to show demonstrated effectiveness.
According to Arkansas, its new application and rating system was approved by OVAE.

Arkansas stated that effectiveness was the most important consideration and was given the
highest point value of the 12 considerations. The E&E calculation for program year 2007-2008
was not completed prior to the 2008-2009 RFP process, but effectiveness was considered in
awarding new grants. A panel read the applications for all eight literacy councils cited in the
draft audit report, and each literacy council met the minimum eligibility score and, therefore, was
awarded the grant for program year 2008-2009. Arkansas reiterated that programs that did not
meet E&E were placed on program improvement plans. As an example, Arkansas provided
information for Monroe County. Arkansas stated that Monroe received a “Does Not Satisfy” for
the effectiveness consideration, but it still received an overall score of 75. A score of 75 met the
required minimum score, so Arkansas awarded Monroe a new grant. Arkansas acknowledged
that, for program year 2007-2008, Monroe had an overall E&E of only 19.6 percent. However,
the score was not available until after program year 2007-2008 (the year the OIG audited).
 
Final Report
ED-OIG/A06J0001                                                                      Page 14 of 43

Arkansas asserted that Monroe was placed on a program improvement plan and was monitored
during program year 2008-2009. Monroe’s preliminary E&E score for program year 2008-2009
was 90.1 percent, which would make it eligible for a continuation award.

Arkansas commented that literacy councils make up a small percentage of students served in
Arkansas, and three literacy councils recently declined the continuation grant because of
increased evaluations and regulations from Arkansas and OVAE. Arkansas believes that it has a
strong measurement of effectiveness for the literacy councils through the E&E calculation and is
unsure how a literacy council or community-based organization can show demonstrated
effectiveness if it does not have students nor does not pre-test or post-test students that have
completed the program.

OIG Response

Arkansas provided evidence showing that it defined effectiveness for the AEFLA program.
Therefore, we adjusted the finding to explain that, even with the established criteria, Arkansas
did not ensure that literacy councils met the eligible provider criteria prior to awarding new
grants for program year 2008-2009. We made some technical changes to the finding for clarity.
We also deleted recommendations to develop a definition of effectiveness and to include specific
information in grant award documents. Further, we added a recommendation that Arkansas
evaluate the effectiveness of literacy councils and provide the Department with documentation
and assurances that demonstrate all literacy councils who received program year 2008-2009
awards met the definition of effectiveness for the prior grant period.

Although Arkansas provided evidence that it has a definition of effectiveness, it did not provide
any additional information that the literacy councils showed “demonstrated effectiveness” prior
to the awarding of new grants for program year 2008-2009. We acknowledge that effectiveness
is only part of the 12 considerations for receiving a grant. However, the AEFLA clearly states
that, to be considered eligible for a grant, a literacy council must show demonstrated
effectiveness. To show the effectiveness of the program, a provider should have students
enrolled in the program and advancing through the program to meet the goals of the AEFLA.

Arkansas revised its application review process for program year 2008-2009, and effectiveness
was reviewed as part of the twelve considerations; however, before receiving grant funds, both
literacy councils and community based organizations must show demonstrated effectiveness.
Had Arkansas completed its E&E calculations prior to awarding grant funds for program year
2008-2009, it would have concluded that Monroe did not meet the effectiveness requirement of
the grant and was not eligible for the grant. Based on its E&E calculation, Arkansas also would
have concluded that six of the other seven literacy councils in our sample (Carroll, Crittenden,
Dogwood, Drew, Grant, and Hot Springs) failed to demonstrate effectiveness for the three years
just prior to receiving new grant funds for program year 2008-2009. At a minimum, Arkansas
should have submitted the seven literacy councils to the State Board of Workforce Education and
Career Opportunities for appropriate action prior to continuation of funding.

Arkansas is correct when it states that the AEFLA requires all agencies to have direct and
equitable access to the grant. However, if literacy councils fail to meet the basic requirement of
the grant (definition of an eligible provider), they should not be eligible for funds. We could find
nothing in the law that would indicate that literacy councils and community-based organizations
 
Final Report
ED-OIG/A06J0001                                                                                  Page 15 of 43

that have not demonstrated their effectiveness may receive Federal funds simply because they
apply for the grant. The law requires only that any provider be allowed to apply.


FINDING NO. 3 – Arkansas and Local Providers Did Not Use Adult
                Education Grant Funds in Compliance with Fiscal
                Requirements
Arkansas providers did not expend adult education grant funds in accordance with applicable
law, regulations, and guidance. We reviewed 4 judgmentally selected months’ worth of
expenditures for 25 randomly selected local providers and 2 State providers.4

According to 34 C.F.R. § 80.22, OMB Circular A-87 sets the Federal principles for determining
allowable costs for grants to a State.5 During program year 2007-2008, Arkansas and its
providers expended $521,259 in adult education funds that did not meet the requirements of
OMB Circular A-87. Two local providers used adult education grant funds for $13,027 in
unallowable costs, and 23 of the 25 local providers did not provide adequate documentation for
$506,423. In addition, the two State providers did not provide adequate documentation for
$1,809 in adult education grant expenditures.

Two local providers charged the grant for $13,027 in unallowable expenses. One local provider
spent $3,338 for carpet in leased space. OMB Circular A-87(B)(15)(b)(3) states that grantees
cannot make improvements that materially increase the value or life of the building with grant
funds. The provider had office space in a leased facility and should not have used grant funds to
upgrade the carpet. The remaining $9,689 was used to pay for GED® testers. Sections 223(a)(9)
and 241(a) of the AEFLA provide that grant funds must be spent only on instructors of adult
education programs. The GED® testers only administer the test and, therefore, should not be
paid with adult education funds. (See Enclosure 1 for excerpts from the AEFLA and
OMB Circular A-87.)

Twenty-three local providers did not provide sufficient documentation to adequately support
$506,423 of grant fund expenditures charged to the adult education grant. OMB Circular
A-87(A)(C)(j) states that the expenditures of the grant must be adequately documented. For all
25 local providers, a majority of the expenditures were for payroll. However, 23 providers did
not obtain self-certifications of time for employees who worked solely on the grant or personnel
activity reports for employees who worked on multiple grants (OMB Circular A-87(B)(8)(h)(3-
5)). Eight providers did not provide adequate support for supplies that were purchased. The
providers did not provide purchase orders, invoices, or receipts to support the purchase.

The two State providers did not adequately document $1,809 of grant fund expenditures. The
grants to the State providers are to be used for leadership training for the program. The
inadequately documented costs were for food, beverages, and items purchased for training.
Items for conferences are allowable per OMB Circular A-87(B)(27); however, the State
providers did not provide adequate documentation to support the purchases. The documentation
                                                            
4
    The months were selected because they were the highest months for expenditures. We selected August 2007,
    October 2007, February 2008, and June 2008.
5
    The cost principles in OMB Circular A-87 are codified in 2 C.F.R. Part 225 (January 1, 2007, version).
 
Final Report
ED-OIG/A06J0001                                                                        Page 16 of 43

should show how the cost is tied to the professional conference. The State providers also
reimbursed attendees’ travel expenses without adequate documentation of hotel and food
expenses. OMB Circular A-87(B)(43) allows cost related to travel to be paid by the grant funds;
however, the expenditures must be adequately documented.

Arkansas did not have an internal control system in place to provide reasonable assurance that
the expenditures of the grant met the requirements of OMB Circular A-87 or other applicable
laws and regulations. When the expenses are reported to the State office, Arkansas verifies only
that teachers are certified to teach. Expenditures are not tested to ensure they meet the
requirements of the grant or are in compliance with the cost principles.

Recommendations

We recommend that the Assistant Secretary, OVAE, require Arkansas to—

3.1   Return to the Department $13,027 in unallowable costs;

3.2   Provide adequate documentation to support $508,232 in inadequately documented costs or
      return the inadequately documented amount to the Department; and

3.3   Provide training to Arkansas officials and local providers to ensure they understand the
      fiscal requirements for the adult education grants.

Auditee Comments

Arkansas agreed with the portion of the finding related to the unallowable cost of $9,689 for
GED® testers and plans to return the funds and provide additional training to staff so that they
are aware that this is not allowed. Arkansas did not agree with the other portions of the finding,
provided additional documentation and retroactive self-certifications of time, and provided
additional receipts and explanations. Arkansas also stated that it plans to provide additional
training for all staff to ensure they understand all requirements of the grant.

OIG Response

We made some changes to the finding based on our review of the additional documentation
provided. Arkansas provided documentation that supported the majority of the originally
inadequately documented expenses for the two State leadership providers. It also provided
policies and procedures for meals during training, which stated that receipts are not required and
the U.S. General Services Administration’s (GSA) per diem rates are used for travel. Therefore,
we adjusted the inadequately documented amount downward from $49,560 to $1,809.

Arkansas provided self-certifications of time for the local providers. However, the
documentation did not demonstrate that the certifications were prepared timely. Arkansas
provided certifications that were prepared well after we requested them and more than a year
after the certification period ended (the self-certifications of time were not signed and dated until
calendar year 2009, even though the work was performed for program year 2007-2008). All of
the employees whose self-certifications we reviewed appeared to have spent 100 percent of their
time on the Adult Education grant, but the certifications were not prepared at least semiannually,
 
Final Report
ED-OIG/A06J0001                                                                       Page 17 of 43

as required by OMB Circular A-87, Attachment B, Paragraph 8.h.(3). Therefore, we have no
assurances of the validity of the self-certifications of time because they were not provided to us
until a year after the initial time sheets were supposed to be submitted. Therefore, we did not
change our finding or recommendations related to the allowability of the payroll expenses.

Arkansas also provided additional documentation, such as invoices, cancelled checks, bank
statements, and accounting records, from local providers to support other costs associated with
the grant that we originally considered inadequately documented. Based on our review of the
additional information, we adjusted our finding downward from $566,815 to $506,423.




 
Final Report
ED-OIG/A06J0001                                                                    Page 18 of 43




                                   OTHER MATTERS


OVAE Reported Findings from Past Program Reviews

OVAE completed two program reviews, one in April 2006 and one in August 2007. For the
program review completed in April 2006, OVAE determined that Arkansas’ Request for
Proposal (RFP) and the Application Rating Sheet did not include language of the 12
considerations that are required by the AEFLA. Arkansas was required to fix the RFP prior to
any new competition and obtain approval from OVAE. During the review, OVAE also noted
that Arkansas did not meet the direct and equitable requirement in the AEFLA. Arkansas did not
hold the bidders’ conference as outlined in the State plan. It also did not have a panel evaluate
and rate the RFPs. Through a misinterpretation of the AEFLA, Arkansas restricted who was
allowed to apply for the grant.

OVAE returned to conduct a second program review during August 2007. The August 2007
program review addressed the items identified in the April 2006 program review. In addition,
OVAE determined that Arkansas did not provide direct and equitable access to applicants for the
E/L Civics grant. The community-based organizations and literacy councils were limited to an
award of $5,000, while local educational agencies and postsecondary educational institutions
were limited to an award of $12,500. Arkansas also limited how community-based organizations
and literacy councils used Federal funds to pay teachers and tutors.

Based on our review of the corrective action plan and the results of our audit work, Arkansas has
completed the actions required for the above-mentioned items. All items have been approved by
OVAE and closed. However, given the issues our audit disclosed, we suggest that OVAE
continue to monitor Arkansas to ensure the application process meets all of the requirements set
forth in the AEFLA.




 
Final Report
ED-OIG/A06J0001                                                                     Page 19 of 43




                  OBJECTIVES, SCOPE, AND METHODOLOGY


The objectives of our audit were to determine whether (1) Arkansas adequately monitored the
performance of providers receiving adult education funds; (2) Arkansas and its providers used
adult education funds in accordance with the law, regulations, and guidance; and (3) Arkansas
awarded adult education grants to providers appropriately given the considerations identified in
Section 231 of the AEFLA. Our audit covered the period July 1, 2007, through June 30, 2008
(program year 2007-2008).

To accomplish our objective, we performed the following procedures.

       Reviewed the internal control structure, policy, procedures, and practices Arkansas used
        to award and expend adult education funds.
       Examined prior reviews conducted by OVAE and reviewed the legislative auditors’ audit
        documentation supporting their OMB Circular A-133 Single Audit Report for the year
        ended June 30, 2007.
       Requested and obtained a list of all local providers that received adult education funding
        for program year 2007-2008.
       Requested and obtained the original applications for 25 randomly selected local providers
        form the universe of 76 (the original applications were for program year 2003-2004).
       Requested and obtained the monthly expense reports for August 2007, October 2007,
        February 2008, and June 2008 for the 25 randomly selected local providers and the only
        two State providers.
       Requested and reviewed the enrolled and advanced data from the NRS to determine
        whether the 25 randomly selected local providers met the established benchmarks for
        program years 2003-2004 through 2007-2008.
       Reviewed monitoring reports, policy, procedures, and established schedule for
        monitoring local providers who received adult education grant funds for program reviews
        conducted in program year 2007-2008.
       Interviewed Arkansas program staff and OVAE officials.

To test whether Arkansas awarded adult education grant funds only to local providers that met
the requirements of the AEFLA, we randomly selected 25 local providers, including 8 literacy
councils, from the universe of 76. Our audit period was program year 2007-2008. However, the
initial applications for the providers receiving adult education grant funds during program year
2007-2008 were received for program year 2003-2004 (the providers were awarded a 5-year
grant through program year 2007-2008). We reviewed all of the grants for the 25 local providers
selected. To test whether the local providers’ programs were effective, we reviewed data
obtained from NRS to ensure that the local providers met the annual benchmarks. We also
reviewed the E&E calculation that Arkansas completed for program year 2007-2008. For the




 
Final Report
ED-OIG/A06J0001                                                                                     Page 20 of 43

eight literacy councils in our sample, we reviewed the indicators and data from NRS to ensure
that the literacy councils met the eligible provider requirement of the AEFLA.6

To test the expenditures of the grant, we reviewed 4 judgmentally selected months’ worth of
expenditures for the 25 randomly selected local providers and 2 State providers. The months
selected were August 2007, October 2007, February 2008, and June 2008, because they were the
months that had the highest amount of expenditures.

To test whether Arkansas adequately monitored the adult education providers, we reviewed the
reports for the 10 monitoring reviews that Arkansas completed during program year 2007-2008.
We reviewed the checklist used for the review as well as the final report submitted to the
providers and the corrective action plans completed.

We conducted our fieldwork at Arkansas’ offices and our office from November 2008 through
May 2009. We held an exit conference with Arkansas officials on September 9, 2009, to discuss
the results of the audit.

We conducted this performance audit in accordance with generally accepted government
auditing standards. Those standards require that we plan and perform the audit to obtain
sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions
based on our audit objectives. We believe that the evidence obtained provides a reasonable basis
for our findings and conclusions based on our audit objectives.




                                                            
6
    We limited testing only to the literacy councils because only literacy councils and community-based organizations
    are required to demonstrate effectiveness to meet the definition of an eligible provider.
 
Final Report
ED-OIG/A06J0001                                                                        Page 21 of 43

    Enclosure 1: Selected Excerpts from the Applicable Law and
                        OMB Circular A-87
Title II, Adult Education and Family Literacy Act, of the Workforce Investment Act
(Public Law 105-220)
   CHAPTERS 1, 2, and 3—Federal Provisions, State Provisions, and Local Provisions

                       CHAPTER 1—FEDERAL PROVISIONS

SEC. 212. PERFORMANCE ACCOUNTABILITY SYSTEM.

(a) Purpose.--The purpose of this section is to establish a comprehensive performance
accountability system, comprised of the activities described in this section, to assess the
effectiveness of eligible agencies in achieving continuous improvement of adult
education and literacy activities funded under this subtitle, in order to optimize the return
on investment of Federal funds in adult education and literacy activities.

(b) Eligible Agency Performance Measures.—
        (1) In general.--For each eligible agency, the eligible agency performance
        measures shall consist of—
          (A)(i) the core indicators of performance described in paragraph (2)(A); and (ii)
                additional indicators of performance (if any) identified by the eligible
                agency under paragraph (2)(B); and (B) an eligible agency adjusted level
                of performance for each indicator described in subparagraph (A).
        (2) Indicators of performance.—
                (A) Core indicators of performance.--The core indicators of performance
                shall include the following:
                        (i) Demonstrated improvements in literacy skill levels in reading,
                        writing, and speaking the English language, numeracy, problem
                        solving, English language acquisition, and other literacy skills.
                        (ii) Placement in, retention in, or completion of, postsecondary
                        education, training, unsubsidized employment or career
                        advancement.
                        (iii) Receipt of a secondary school diploma or its recognized
                        equivalent.
                (B) Additional indicators.--An eligible agency may identify in the State
                plan additional indicators for adult education and literacy activities
                authorized under this subtitle.
        (3) Levels of performance.—
                (A) Eligible agency adjusted levels of performance for core indicators.—
                        (i) In general.--For each eligible agency submitting a State plan,
                        there shall be established, in accordance with this subparagraph,
                        levels of performance for each of the core indicators of
                        performance described in paragraph (2)(A) for adult education and
                        literacy activities authorized under this subtitle. The levels of



 
Final Report
ED-OIG/A06J0001                                                                    Page 22 of 43

                  performance established under this subparagraph shall, at a
                  minimum—
                           (I) be expressed in an objective, quantifiable, and
                           measurable form; and
                           (II) show the progress of the eligible agency toward
                           continuously improving in performance.
                  (ii) Identification in state plan.--Each eligible agency shall identify,
                  in the State plan submitted under section 224, expected levels of
                  performance for each of the core indicators of performance for the
                  first 3 program years covered by the State plan.
                  (iii) Agreement on eligible agency adjusted levels of performance
                  for first 3 years.--In order to ensure an optimal return on the
                  investment of Federal funds in adult education and literacy
                  activities authorized under this subtitle, the Secretary and each
                  eligible agency shall reach agreement on levels of performance for
                  each of the core indicators of performance, for the first 3 program
                  years covered by the State plan, taking into account the levels
                  identified in the State plan under clause (ii) and the factors
                  described in clause (iv). The levels agreed to under this clause shall
                  be considered to be the eligible agency adjusted levels of
                  performance for the eligible agency for such years and shall be
                  incorporated into the State plan prior to the approval of such plan.
                  (iv) Factors.--The agreement described in clause (iii) or (v) shall
                  take into account—
                           (I) how the levels involved compare with the eligible
                           agency adjusted levels of performance established for other
                           eligible agencies, taking into account factors including the
                           characteristics of participants when the participants entered
                           the program, and the services or instruction to be provided;
                           and
                           (II) the extent to which such levels involved promote
                           continuous improvement in performance on the
                           performance measures by such eligible agency and ensure
                           optimal return on the investment of Federal funds.
                  (v) Agreement on eligible agency adjusted levels of performance
                  for 4th and 5th years.-- Prior to the fourth program year covered by
                  the State plan, the Secretary and each eligible agency shall reach
                  agreement on levels of performance for each of the core indicators
                  of performance for the fourth and fifth program years covered by
                  the State plan, taking into account the factors described in clause
                  (iv). The levels agreed to under this clause shall be considered to
                  be the eligible agency adjusted levels of performance for the
                  eligible agency for such years and shall be incorporated into the
                  State plan.
                  (vi) Revisions.--If unanticipated circumstances arise in a State
                  resulting in a significant change in the factors described in clause


 
Final Report
ED-OIG/A06J0001                                                                      Page 23 of 43

                       (iv)(II), the eligible agency may request that the eligible agency
                       adjusted levels of performance agreed to under clause (iii) or (v) be
                       revised. The Secretary, after collaboration with the representatives
                       described in section 136( j), shall issue objective criteria and
                       methods for making such revisions.
               (B) Levels of performance for additional indicators.--The eligible agency
               may identify, in the State plan, eligible agency levels of performance for
               each of the additional indicators described in paragraph (2)(B). Such
               levels shall be considered to be eligible agency adjusted levels of
               performance for purposes of this subtitle.
(c) Report.--
       (1) In general.--Each eligible agency that receives a grant under section 211(b)
       shall annually prepare and submit to the Secretary a report on the progress of the
       eligible agency in achieving eligible agency performance measures, including
       information on the levels of performance achieved by the eligible agency with
       respect to the core indicators of performance.
         (2) Information dissemination.--The Secretary--
         (A) shall make the information contained in such reports available to the
               general public through publication and other appropriate methods;
         (B) shall disseminate State-by-State comparisons of the information; and
         (C) shall provide the appropriate committees of Congress with copies of such
               reports.

                         CHAPTER 2—STATE PROVISIONS

SEC. 223. STATE LEADERSHIP ACTIVITIES.

(a) In General.--Each eligible agency shall use funds made available under section
222(a)(2) for one or more of the following adult education and literacy activities:
       (1) The establishment or operation of professional development programs to
       improve the quality of instruction provided pursuant to local activities required
       under section 231(b), including instruction incorporating phonemic awareness,
       systematic phonics, fluency, and reading comprehension, and instruction provided
       by volunteers or by personnel of a State or outlying area.
       (2) The provision of technical assistance to eligible providers of adult education
       and literacy activities.
       (3) The provision of technology assistance, including staff training, to eligible
       providers of adult education and literacy activities to enable the eligible providers
       to improve the quality of such activities.
       (4) The support of State or regional networks of literacy resource centers.
       (5) The monitoring and evaluation of the quality of, and the improvement in, adult
       education and literacy activities.
       (6) Incentives for--
               (A) program coordination and integration; and
               (B) performance awards.



 
Final Report
ED-OIG/A06J0001                                                                        Page 24 of 43

       (7) Developing and disseminating curricula, including curricula incorporating
       phonemic awareness, systematic phonics, fluency, and reading comprehension.
       (8) Other activities of statewide significance that promote the purpose of this title.
       (9) Coordination with existing support services, such as transportation, child care,
       and their assistance designed to increase rates of enrollment in, and successful
       completion of, adult education and literacy activities, to adults enrolled in such
       activities.
        (10) Integration of literacy instruction and occupational skill training, and
       promoting linkages with employers.
       (11) Linkages with postsecondary educational institutions.

(b) Collaboration.--In carrying out this section, eligible agencies shall collaborate where
possible, and avoid duplicating efforts, in order to maximize the impact of the activities
described in subsection (a).

(c) State-Imposed Requirements.--Whenever a State or outlying area implements any rule
or policy relating to the administration or operation of a program authorized under this
subtitle that has the effect of imposing a requirement that is not imposed under Federal
law (including any rule or policy based on a State or outlying area interpretation of a
Federal statute, regulation, or guideline), the State or outlying area shall identify, to
eligible providers, the rule or policy as being State- or outlying area-imposed.

SEC. 224. STATE PLAN.

(a) 5-YEAR PLANS.—
        (1) IN GENERAL.—Each eligible agency desiring a grant
under this subtitle for any fiscal year shall submit to, or have
on file with, the Secretary a 5-year State plan.
        (2) COMPREHENSIVE PLAN OR APPLICATION.—The eligible
agency may submit the State plan as part of a comprehensive
plan or application for Federal education assistance.

(b) PLAN CONTENTS.—In developing the State plan, and any revisions to the State
plan, the eligible agency shall include in the State plan or revisions—
        (1) an objective assessment of the needs of individuals in the State or outlying
        area for adult education and literacy activities, including individuals most in need
        or hardest to serve;
        (2) a description of the adult education and literacy activities that will be carried
        out with any funds received under this subtitle;
        (3) a description of how the eligible agency will evaluate annually the
        effectiveness of the adult education and literacy activities based on the
        performance measures described in section 212;
        (4) a description of the performance measures described in section 212 and how
        such performance measures will ensure the improvement of adult education and
        literacy activities in the State or outlying area;



 
Final Report
ED-OIG/A06J0001                                                                      Page 25 of 43

       (5) an assurance that the eligible agency will award not less than one grant under
       this subtitle to an eligible provider who offers flexible schedules and necessary
       support services (such as child care and transportation) to enable individuals,
       including individuals with disabilities, or individuals with other special needs, to
       participate in adult education and literacy activities, which eligible provider shall
       attempt to coordinate with support services that are not provided under this
       subtitle prior to using funds for adult education and literacy activities provided
       under this subtitle for support services;
       (6) an assurance that the funds received under this subtitle will not be expended
       for any purpose other than for activities under this subtitle;
       (7) a description of how the eligible agency will fund local activities in
       accordance with the considerations described in section 231(e);
       (8) an assurance that the eligible agency will expend the funds under this subtitle
       only in a manner consistent with fiscal requirements in section 241;
       (9) a description of the process that will be used for public participation and
       comment with respect to the State plan;
       (10) a description of how the eligible agency will develop program strategies for
       populations that include, at a minimum—
               (A) low-income students;
               (B) individuals with disabilities;
               (C) single parents and displaced homemakers; and
               (D) individuals with multiple barriers to educational
               enhancement, including individuals with limited English
               proficiency;
       (11) a description of how the adult education and literacy activities that will be
       carried out with any funds received under this subtitle will be integrated with
       other adult education, career development, and employment and training activities
       in the State or outlying area served by the eligible agency; and
       (12) a description of the steps the eligible agency will take to ensure direct and
       equitable access, as required in section 231(c)(1).

(c) PLAN REVISIONS.—When changes in conditions or other factors require substantial
revisions to an approved State plan, the eligible agency shall submit the revisions to the
State plan to the Secretary.

(d) CONSULTATION.—The eligible agency shall—
      (1) submit the State plan, and any revisions to the State plan, to the Governor of
      the State or outlying area for review and comment; and
      (2) ensure that any comments by the Governor regarding the State plan, and any
      revision to the State plan, are submitted to the Secretary.

(e) PEER REVIEW.—The Secretary shall establish a peer review process to make
recommendations regarding the approval of State plans.

(f ) PLAN APPROVAL.—A State plan submitted to the Secretary shall be approved by
the Secretary unless the Secretary makes a written determination, within 90 days after


 
Final Report
ED-OIG/A06J0001                                                                          Page 26 of 43

receiving the plan, that the plan is inconsistent with the specific provisions of this
subtitle.


                         CHAPTER 3—LOCAL PROVISIONS

SEC. 231. GRANTS AND CONTRACTS FOR ELIGIBLE PROVIDERS.

(a) Grants and Contracts.--From grant funds made available under section 211(b), each
eligible agency shall award multiyear grants or contracts, on a competitive basis, to
eligible providers within the State or outlying area to enable the eligible providers to
develop, implement, and improve adult education and literacy activities within the State.

(b) Required Local Activities.--The eligible agency shall require that each eligible
provider receiving a grant or contract under subsection (a) use the grant or contract to
establish or operate one or more programs that provide services or instruction in one or
more of the following categories:
        (1) Adult education and literacy services, including workplace literacy services.
        (2) Family literacy services.
        (3) English literacy programs.

(c) Direct and Equitable Access; Same Process.--Each eligible agency receiving funds
under this subtitle shall ensure that--
        (1) all eligible providers have direct and equitable access to apply for grants or
        contracts under this section; and
        (2) the same grant or contract announcement process and application process is
        used for all eligible providers in the State or outlying area.

(d) Special Rule.--Each eligible agency awarding a grant or contract under this section
shall not use any funds made available under this subtitle for adult education and literacy
activities for the purpose of supporting or providing programs, services, or activities for
individuals who are not individuals described in subparagraphs (A) and (B) of section
203(1), except that such agency may use such funds for such purpose if such programs,
services, or activities are related to family literacy services. In providing family literacy
services under this subtitle, an eligible provider shall attempt to coordinate with programs
and services that are not assisted under this subtitle prior to using funds for adult
education and literacy activities under this subtitle for activities other than adult
education activities.

(e) Considerations.--In awarding grants or contracts under this section, the eligible
agency shall consider--
       (1) the degree to which the eligible provider will establish measurable goals for
       participant outcomes;
       (2) the past effectiveness of an eligible provider in improving the literacy skills of
       adults and families, and, after the 1-year period beginning with the adoption of an
       eligible


 
Final Report
ED-OIG/A06J0001                                                                    Page 27 of 43

     agency's performance measures under section 212, the success of an eligible
     provider receiving funding under this subtitle in meeting or exceeding such
     performance measures, especially with respect to those adults with the lowest
     levels of literacy;
     (3) the commitment of the eligible provider to serve individuals in the community
     who are most in need of literacy services, including individuals who are low-
     income or have
     minimal literacy skills;
     (4) whether or not the program--
             (A) is of sufficient intensity and duration for participants to achieve
             substantial learning gains; and
             (B) uses instructional practices, such as phonemic awareness, systematic
             phonics, fluency, and reading comprehension that research has proven to
             be effective
             in teaching individuals to read;
     (5) whether the activities are built on a strong foundation of research and effective
     educational practice;
     (6) whether the activities effectively employ advances in technology, as
     appropriate, including the use of computers;
     (7) whether the activities provide learning in real life contexts to ensure that an
     individual has the skills needed to compete in the workplace and exercise the
     rights and
     responsibilities of citizenship;
     (8) whether the activities are staffed by well-trained instructors, counselors, and
     administrators;
     (9) whether the activities coordinate with other available resources in the
     community, such as by establishing strong links with elementary schools and
     secondary schools, postsecondary educational institutions, one-stop centers, job
     training programs, and social service agencies;
     (10) whether the activities offer flexible schedules and support services (such as
     child care and transportation) that are necessary to enable individuals, including
     individuals with disabilities or other special needs, to attend and complete
     programs;
     (11) whether the activities maintain a high-quality information management
     system that has the capacity to report participant outcomes and to monitor
     program performance against the eligible agency performance measures; and
     (12) whether the local communities have a demonstrated need for additional
     English literacy programs.




 
Final Report
ED-OIG/A06J0001                                                                       Page 28 of 43


                                 OMB Circular A-87 (5/10/2004)

Attachment (A)

C. Basic Guidelines

1. Factors affecting allowability of costs. To be allowable under Federal awards, costs
must meet the following general criteria:

       a. Be necessary and reasonable for proper and efficient performance and
       administration of Federal awards.

       b. Be allocable to Federal awards under the provisions of this Circular.

       c. Be authorized or not prohibited under State or local laws or regulations.

       d. Conform to any limitations or exclusions set forth in these principles, Federal
       laws, terms and conditions of the Federal award, or other governing regulations as
       to types or amounts of cost items.

       e. Be consistent with policies, regulations, and procedures that apply uniformly to
       both Federal awards and other activities of the governmental unit.

       f. Be accorded consistent treatment. A cost may not be assigned to a Federal
       award as a direct cost if any other cost incurred for the same purpose in like
       circumstances has been allocated to the Federal award as an indirect cost.

       g. Except as otherwise provided for in this Circular, be determined in accordance
       with generally accepted accounting principles.

       h. Not be included as a cost or used to meet cost sharing or matching requirements
       of any other Federal award in either the current or a prior period, except as
       specifically provided by Federal law or regulation.

       i. Be the net of all applicable credits.

       j. Be adequately documented.

Attachment (B)

(8) Compensation for personal services:
             (h) (3) Where employees are expected to work solely on a single
             Federal award or cost objective, charges for their salaries and
             wages will be supported by periodic certifications that the
             employees worked solely on that program for the period covered
             by the certification. These certifications will be prepared at least
             semi-annually and will be signed by the employee or supervisory


 
Final Report
ED-OIG/A06J0001                                                                         Page 29 of 43

               official having firsthand knowledge of the work performed by the
               employee.

               (4) Where employees work on multiple activities or cost
               objectives, a distribution of their salaries or wages will be
               supported by personnel activity reports or equivalent
               documentation which meets the standards in subsection (5) unless
               a statistical sampling system (see subsection (6)) or other substitute
               system has been approved by the cognizant Federal agency. Such
               documentary support will be required where employees work on:

                       (a) More than one Federal award,
                       (b) A Federal award and a non-Federal award,
                       (c) An indirect cost activity and a direct cost activity,
                       (d) Two or more indirect activities which are allocated
                       using different allocation bases, or
                       (e) An unallowable activity and a direct or indirect cost
               activity.

               (5) Personnel activity reports or equivalent documentation must
               meet the following standards:
                       (a) They must reflect an after-the-fact distribution of the
                       actual activity of each employee,
                       (b) They must account for the total activity, for which each
                       employee is compensated,
                       (c) They must be prepared at least monthly and must
                       coincide with one or more pay periods, and
                       (d) They must be signed by the employee.

(15) Equipment and other capital expenditures.
             (b) The following rules of allowability shall apply to equipment and other
             capital expenditures:
             (3) Capital expenditures for improvements to land, buildings, or
             equipment which materially increase their value or useful life are
             unallowable as a direct cost except with the prior approval of the awarding
             agency.

(27) Meetings and conferences: Cost of meetings and conferences, the primary purpose
of which is the dissemination of technical information, are allowable. This includes costs
of meals, transportation, rental facilities, speakers’ fees, and other items incidental to
such meetings or conferences.

(43) Travel costs:
               a. General. Travel costs are the expenses for transportation, lodging,
               subsistence, and related items incurred by employees who are in travel
               status on official business of the governmental unit. Such costs may be


 
Final Report
ED-OIG/A06J0001                                                                 Page 30 of 43

           charged on an actual cost basis, on a per diem or mileage basis in lieu of
           actual costs incurred, or on a combination of the two, provided the method
           used is applied to an entire trip and not to selected days of the trip, and
           results in charges consistent with those normally allowed in like
           circumstances in the governmental unit’s non-federally-sponsored
           activities. Notwithstanding the provisions of Attachment B, section 19,
           General government expenses, travel costs of officials covered by that
           section are allowable with the prior approval of an awarding agency when
           they are specifically related to Federal awards.

           b. Lodging and subsistence. Costs incurred by employees and officers for
           travel, including costs of lodging, other subsistence, and incidental
           expenses, shall be considered reasonable and allowable only to the extent
           such costs do not exceed charges normally allowed by the governmental
           unit in its regular operations as the result of the governmental unit’s
           written travel policy. In the absence of an acceptable, written
           governmental unit policy regarding travel costs, the rates and amounts
           established under subchapter I of Chapter 57, Title 5, United States Code
           (“Travel and Subsistence Expenses; Mileage Allowances”), or by the
           Administrator of General Services, or by the President (or his or her
           designee) pursuant to any provisions of such subchapter shall apply to
           travel under Federal awards (48 CFR 31.205-46(a)).




 
Final Report
ED-OIG/A06J0001                                   Page 31 of 43




                  Enclosure 2: Auditee Comments




 
Final Report
ED-OIG/A06J0001                                                                       Page 32 of 43

                                             STATE OF ARKANSAS 




  Mike Beebe                                                                William L. “Bill” Walker, Jr. 
                                                                  f
October 30, 2009


Mr. Gary D. Whitman, Regional Inspector General of Audit
US Department of Education
Office of Inspector General
1999 Bryant Street, Suite 1440
Dallas, Texas 75201

Dear Mr. Whitman:

The enclosed response contains support documentation related to the draft audit
report (Control Number ED-OIG/A06J0001) received by the Arkansas
Department of Career Education on October 5, 2009. The Arkansas Department
of Career Education’s Adult Education Division has responded to the first two
findings and provided clarification for acronyms, abbreviations, and additional
background support for the following:

        1.      Arkansas did not adequately monitor providers
        2.      Arkansas has not defined demonstrated effectiveness for literacy
                councils

The Arkansas Department of Career Education’s Finance Office will provide
adequate documentation to support the inadequately documented costs
mentioned in the draft report. The Finance Office will also provide training to
Arkansas officials and local providers to ensure they understand fiscal
requirements for the adult education grants listed in the third finding.

         3. Arkansas and local providers did not use adult education grant funds
            in compliance with fiscal requirements.

Please review the attached documentation provided by this Department. We
await your final review and appreciate being given an opportunity to provide
additional information to support a positive outcome as a result of this program
review.


 
Final Report
ED-OIG/A06J0001                                                                    Page 33 of 43


              Three Capitol Mall      Little Rock, AR 72201     (501) 682-1500
                    http://ace.arkansas.gov An Equal Opportunity Employer

Thank you for the professionalism of the reviewing team and their willingness to
address needed areas of program improvement. Please contact our office for
additional clarification or assistance in this matter.

                                      Sincerely,

                                      /s/

                              William L. “Bill” Walker, Jr.

                              attachments




 
Final Report
ED-OIG/A06J0001                                                                      Page 34 of 43

               Subject: Effective and Efficient Evaluation Calculations


POLICY: According to Arkansas Act 1280 of 2007, Section 30, ”All funds that become
available for adult education shall first be distributed to those administrative units
determined to be operating efficient and effective adult education programs, under
criteria established by the State Board of Workforce Education and Career Opportunities.
The criteria shall include the relative efficiency of administration of the program in the
counties served and achievement of the federal performance indicators.”

To be considered effective and efficient, local adult education programs and literacy
councils receiving adult education funds will be required to meet or exceed 80% of the
negotiated core indicators of performance. Programs not meeting 80% of the negotiated
core indicators of performance in the first year will be required to submit an improvement
plan. The improvement plan will address each core indicator of performance that was not
met and will include measurable objectives for correcting deficiencies during the
following year. State program advisors will provide technical assistance to each program
submitting an improvement plan and closely monitor programs during the year. Core
indicators of performance are defined in the federal Workforce Investment Act of 1998,
Title II, the Adult Education and Family Literacy Act (AEFLA); however, using the core
indicators as criteria for effective and efficient evaluations and imposing an 80%
minimum score is a state decision.

Programs that do not meet at least 80% of their negotiated core indicators of performance
for a second consecutive year will be assigned a state technical assistance team to assist
the program in addressing its deficiencies. The team members may include but not be
limited to state and local staff, administrators, teachers and students. Quarterly progress
reports will be submitted by the program being monitored to the Adult Education
Division describing the progress being made to bring the program into compliance with
core indicators. Programs not achieving 80% of their negotiated core indicators of
performance for three consecutive years will be submitted to the State Board of
Workforce Education and Career Opportunities for appropriate action prior to
continuation of funding.


INFORMATION/RATIONALE: Relevant sections of law and regulation: Arkansas Act
1280 of 2007, Section 30; and federal Workforce Investment Act of 1998 (P.L. 105-220),
Title II, The Adult Education and Family Literacy Act (AEFLA), Section 212 (b).

EFFECTIVE DATE: JULY 1, 1991

REVISED DATE: SEPTEMBER 13, 1993
REVISED DATE: APRIL 18, 2003
REVISED DATE: NOVEMBER 15, 2008




 
Final Report
ED-OIG/A06J0001                                                                   Page 35 of 43

Response to the Draft Report of the Office of the Inspector General, U.S.
Department of Education (Control Number ED-OIG/A06J0001)

Acronyms/Abbreviations and Background Section

Before the main findings are addressed below, four points from the introduction of the
Draft Report should be re-examined. (1) On the Acronyms/Abbreviations page which
precedes the Executive Summary, the acronym GED® stands for General Educational
Development, and is used by the publisher, American Council on Education, to refer to
either the GED tests or GED diploma or credential. The correct acronym EL/Civics
refers to the federal program English Literacy and Civics Education. The name of the
department is now the Arkansas Department of Career Education. NRS stands for the
National Reporting System for Adult Education. (2) In the Background section, on page
3, the Draft Report states, “For program years 2003-2004 through 2007-2008, all local
providers were required to enter into the National Reporting System for Adult Education
(NRS) the number of adults enrolled and advanced in each core indicator.” The NRS is
the national system in which all states report their cumulative data. Local providers do
not enter data into the NRS. Local providers enter their data into the state’s student
management information system which is then used to generate cumulative state reports
for the NRS. The state system is also capable of producing NRS tables for the local
programs. (3) Page 3 of the Background section (also found on page 5 of the Draft
Report) states, “Beginning with program year 2001-2002 and through the current
program year (2008-2009), on an annual basis, Arkansas negotiates with OVAE
benchmarks for the 12 indicators for the adult education grant.” Arkansas, and all other
states, negotiates with OVAE for 11 educational outcome benchmarks (High ASE is not
included) plus four follow-up outcome benchmarks for a total of 15 benchmarks. (4)
Background section, page 3 continues, “Arkansas agreed to meet 80 percent of the
negotiated benchmark for each indicator. OVAE holds Arkansas to these benchmarks,
and the requirements are passed down to the local providers.” Arkansas, as a state,
agrees (and attempts) to meet 100 percent of the benchmarks with OVAE. OVAE does
not require local programs nor has it had states require local programs to meet the
benchmarks. The state policy of “Effective and Efficient Evaluation Calculations”
requires that local programs meet 80 percent of the negotiated benchmarks to be
considered effective and efficient and eligible for funding.


Finding 1: Arkansas did not adequately monitor providers

On July 1, 2000, the National Reporting System for Adult Education (NRS) of the Office
of Vocational and Adult Education (OVAE), Division of Adult Education and Literacy
(DAEL) effectively became the system of accountability in adult education. Prior to
2000, the Arkansas Adult Education Division (AED) had used Quality Indicators of
Performance to do program reviews and used a different set of criteria to do an annual
quantitative report on each local program, called Effective and Efficient (E&E) reports.
The meaning of “Effective and Efficient” is based on Arkansas Act 1280 of 2007 and the
State Board of Career Education policy that requires each local program to meet or


 
Final Report
ED-OIG/A06J0001                                                                       Page 36 of 43

exceed 80 percent of the federally negotiated core indicators of performance (see
attachment of policy “Effective and Efficient Evaluation Calculations”).

According to the E&E policy, programs not meeting or exceeding 80 percent of the
negotiated core indicators in the first year are required to submit a Program Improvement
Plan which addresses each indicator not met and includes measurable objectives for
correcting deficiencies during the following year. Program Advisors from the state office
provide technical assistance and closely monitor programs during the year. Programs
that do not meet at least 80 percent of the negotiated core indicators for a second year are
assigned a state technical assistance team to assist the program in addressing the
deficiencies. Quarterly progress reports are submitted to the state office describing the
progress being made to bring the program into compliance with core indicators.
Programs not achieving 80 percent of the core indicators for three consecutive years are
submitted to the State Board of Career Education for appropriate action prior to
continuation of funding.

In program year 2000-2001, with the full implementation of NRS, a new web-based
management information system needed to be built to process the new student data to be
reported on the required NRS Tables. According to the Arkansas State Unified Plan,
“The Department of Workforce Education (DWE) will evaluate annually the
effectiveness of adult education and literacy activities based on the core indicators of
performance of this State Plan that complies with Section 212 of the Workforce
Investment Act of 1998 (Public Law 105-220). Data will be provided through the
management information system (MIS) to verify that the core indicators are being met
according to the guidelines indicated in the State Plan…. New MIS software is currently
being developed for implementation in July 2000. The new MIS will be aligned with the
National Reporting System for Adult Education and the WIA. It will have the capability
to convert data from the previous system.” (Part II-K-12)

AED decided to discontinue using its outdated system (STUREC) to pursue this new
web-based MIS that would meet the NRS needs. AED had a contract with the Arkansas
Department of Information Services, but after two years of building a system that was
never implemented (AEMIS), the cost required to maintain the system proved to be
unaffordable. Another contract was established with a new company (G*STARS) that
had done work with One-Stop Centers in another state. More than two long years later, a
system was still not in place that would satisfy the requirements of the NRS reports.
Finally, in July 2005, a system was implemented in Arkansas (AERIS) which would meet
the demands of NRS reporting. Between 2001 and 2005, AED struggled with three
different unsuccessful systems and was forced to rely on local programs to keep their
own student data records in their own student management systems (Excel, for example).
The local programs reported their student data to the state office which in turn aggregated
the data to report to OVAE via the NRS.

When the new system (AERIS) was finally implemented, AED was able to place more
confidence in the data from the local programs. In the same year, 2005, AED introduced
a model for E&E calculations which used as its basis the core indicators of performance


 
Final Report
ED-OIG/A06J0001                                                                       Page 37 of 43

benchmarks from the NRS Educational Functioning Levels (EFL) and the four follow-up
goals (see attachment “2008-2009 Effective and Efficient Calculation: Sample Program
X”). The Arkansas E&E calculation model has been presented at a regional NRS
seminar where it received positive reviews from state directors and DAEL staff. In the
calculation, each EFL and goal is weighted (depending on the number of students in it),
so that each EFL or goal carries the weight of the number of students in that level or with
that goal. In this way, a level or goal with only a few students will not unfairly outbalance
a level or goal with many students. The new E&E was introduced to the local program
administrators, and it was decided that program year 2006-2007 would be a transition
year and programs would be held harmless. AED would track each program’s E&E for
2006-2007, but the state policy would not be enforced until program year 2007-2008. In-
depth training sessions were held for all local program administrators.

At the end of program year 2007-2008, E&E calculations were computed, and 28 adult
education and literacy programs had not met the E&E benchmarks. The state office
conducted site reviews of these programs and Program Improvement Plans were put into
effect. During 2008-2009, these 28 programs were monitored closely for progress toward
meeting the federal benchmarks. Based on the preliminary data from 2008-2009 (lacking
one more data match for the post-secondary education goal), it appears that there are only
two programs that are not E&E for 2008-2009. The E&E process has proven to be very
successful in raising programs’ performance, and the results have exceeded our hoped-for
expectations.

When the federal review team from the (DAEL) visited Arkansas in April 2006, one of
their recommendations (not a finding) was to “strengthen the local onsite monitoring
process.” The Adult Education Division (AED) followed through on this
recommendation by adopting a model based on the DAEL Corrective Action Plan for its
on-site in-depth program reviews. This process added timelines and closure procedures
that are essential to continuous program improvement at the local level. Several
questions on the program review address the core indicators of performance benchmarks
and program effectiveness. In addition, the state office integrates the E&E policy into its
in-depth program reviews by choosing to visit those programs with low E&E results.
Finally, during program year 2008-2009, the state office staff made site visits to 100
percent of the adult education and literacy programs statewide.

In conclusion, in our opinion, AED monitors all providers to assure compliance with
applicable federal requirements and assures that performance goals are being achieved.
AED reviews each core indicator of performance for each local provider with a robust
evaluation calculation. AED has a state E&E policy, approved by OVAE, which
implements a process of corrective action for providers that are not meeting the required
benchmarks.




 
Final Report
ED-OIG/A06J0001                                                                       Page 38 of 43

Finding 2: Arkansas has not defined demonstrated effectiveness for literacy councils

There is considerable overlap between Finding 1 and Finding 2 because both are
concerned with the concept of effectiveness. Therefore, the comments in Finding 1
related to program effectiveness should also be considered in the response to Finding 2.
In the case of Finding 2, the effectiveness is specifically focused on the definition of an
“eligible provider” in WIA, Title II, Section 203 (5) (C), “a volunteer literacy
organization of demonstrated effectiveness.” It should be noted here that all of the
literacy councils in Arkansas also have 501(c)3 status as private non-profit agencies, as
found in Section 203 (5) (E).

The Adult Education Division grappled with the definition of demonstrated effectiveness
when it was designing its new competition for federal grants in the spring of 2008 for the
program year 2008-2009. Following the finding by OVAE concerning the 2003-2004
grant award, Arkansas was required to redo its entire RFP and grant process. This is
covered in detail in the Draft Report on page 10, “Other Matters.”

AED was in constant contact with OVAE about the development of the new RFP, and all
documents and procedures were approved by OVAE before implementation. The
application questions and a rating sheet were designed to correlate with all of the 12
considerations that are required by AEFLA. Consideration 2 is “the past effectiveness of
an eligible provider in improving the literacy skills of adults and families.” AED
requested clarification from OVAE about past effectiveness and demonstrated
effectiveness and was instructed that any volunteer literacy organization should have
direct and equitable access to apply for the funds, whether it had a literacy program in
place or not, whether it had students or not. We were instructed to allow all programs to
apply for funds. We were also instructed to add pages to the RFP for those programs that
might not have NRS tables or choose not to show NRS tables in order to document
student data, including educational gains.

The RFP rating sheet was also approved by OVAE, with various weights assigned to
each consideration. Using other states’ rating sheets as examples, and because AED
believed that effectiveness was the most important consideration, Consideration 2 was
given the highest point rating of 15 points out of 100. The state had a reading panel that
rated each applicant’s responses to the 12 considerations, and the eight literacy councils
mentioned in the Draft Report all received enough points to be awarded grants. Monroe
County Literacy Council for example, received the lowest rating possible on
Consideration 2, “Does Not Satisfy,” yet in total received 75 points, enough to qualify for
a grant award.

To continue with the example, Monroe County Literacy Council’s final E&E for program
year 2007-2008, which could not be calculated until after June 30, 2008, was 19.6%.
Because of the low rating, the council was required to submit a Program Improvement
Plan and was monitored throughout program year 2008-2009. One of the reasons their
E&E was so low was because they were not doing a proficient job of post-testing their
students. Working with volunteer tutors presents many difficulties for literacy councils,


 
Final Report
ED-OIG/A06J0001                                                                      Page 39 of 43

and one of those challenges is to educate tutors about NRS regulations and the need to
post-test students when it is time. The preliminary results for 2008-2009 show that
Monroe County Literacy Council’s E&E is now 90.1%. The program is cognizant of the
benchmarks it must meet and the ramifications if those are not met.

Literacy councils make up a small percentage of all students served in the state, yet they
provide a much needed service to small rural Arkansas communities where illiteracy is
very prevalent. These councils work with the hardest to serve individuals and those most
in need, including people at the lowest proficiency levels, people with multiple barriers to
learning, and people with disabilities. It is unfortunate for Arkansas that three literacy
councils have declined to accept their continuation grant awards of federal funds for
program year 2009-2010 due to the increased evaluations and regulations from AED and
OVAE. This will certainly have a negative impact on the communities that these
councils serve.

In conclusion, in our opinion, AED has a quantitative measurement of program
effectiveness through its E&E evaluation calculations. It is not clear how a community-
based organization or volunteer literacy organization can demonstrate effectiveness if it
does not have students, or does not pre-test and post-test students. We seek guidance
from OVAE on that measure. During the program year 2008-2009 RFP process, the E&E
evaluation calculations had not been completed for program year 2007-2008; however,
demonstrated effectiveness was a consideration on the RFP and programs were rated
according to the information provided on their applications. Programs that do not meet
E&E are monitored throughout the program year on a quarterly basis at least. However,
it should be noted that most programs cannot meet E&E early in the year, after the first
quarter, for the basic reason that the students have not had sufficient hours in order to be
post-tested or they have not exited and achieved their goals yet. The E&E process clearly
defines the benchmarks that will be used to assess program effectiveness and carries
powerful consequences for non-compliance that all programs now fully understand.


Finding 3: Arkansas and local providers did not use adult education grant funds in
compliance with fiscal requirements

Included with this response is a detailed documentation for most of the cost cited as
inadequately documented. In the case of the items cited for the Arkansas Literacy
Council and the Arkansas Adult Learning Resource Center two major items of
misunderstanding need to be resolved.

In the first instance neither of these “State” providers requires receipts for meal
reimbursement for the employees or training participants. Both providers follow the
Department of Career Education guidelines (sanctioned by the Official State of Arkansas
Travel Policy) which is essentially the most current GSA Travel guidelines for Arkansas.
In all cases if there is a deviation from the GSA guidelines it will be more stringent.




 
Final Report
ED-OIG/A06J0001                                                                      Page 40 of 43

Secondly the Arkansas Adult Learning Resource Center is not an independent entity. It
is an operational unit of the Southeast Arkansas Education Service Cooperative located at
Monticello, Arkansas. That site is the legal and fiscal center for the Arkansas Adult
Learning Resource Center and is the storehouse for auditable records. Most of the
documents included in this package were readily obtained from this source.

The documents relating to the local providers related mainly to self-certifications of time
or personal activity reports. All the entities have provided after-the-fact certifications
(signed in 2009) for those employees they indicate worked solely for Adult Education
programs and time sheets for those who they indicated worked for multiple grants.

The documents are arranged in the following manner:
      1. Arkansas Literacy Council
      2. Arkansas Adult Learning Resource Center
           (a) August 2007
           (b) October 2007
           (c) February 2008
           (d) June 2008

       3. Local Providers
            (a) Van Buren School District
            (b) Warren School District
            (c) UAM Crossett
            (d) Pulaski Co School District
            (e) Northwest Technical Institute
            (f) Shorter College
            (g) South Arkansas Community College
            (h) National Park Community College – no findings
            (i) Grant Co Literacy Council
            (j) Monroe Co Literacy Council
            (k) Hot Spring Literacy Council
            (l) Hope School District
            (m) Fort Smith School District
            (n) Little Rock School District
            (o) Little Rock School District – AETN
            (p) Little Rock School District – School for Deaf
            (q) Crawford Co Literacy Council
            (r) Dogwood Literacy Council
            (s) Carroll Co Literacy Council
            (t) Crittenden Co Literacy Council
            (u) Arkansas Department of Corrections
            (v) East Arkansas Community College
            (w) Drew Co Literacy Council
            (x) ASU-Searcy
            (y) Arkadelphia School District



 
Final Report
ED-OIG/A06J0001                                                                     Page 41 of 43

The agency concurs with one specific finding that of the unallowable cost of $9,688.50
paid in salary and matching for a GED instructor in the Little Rock School District. This
is against state rule as well as federal policy. The policy and rule will be emphasized at
future training sessions this fiscal year. Recovery of the funds in question will be made
upon completion of this audit process.

The agency does not concur with other financial findings. The documents submitted are
in rebuttal or explanation of these findings and we await your review and response.

The agency has met with local providers on several occasions to discuss these issues. In
addition, Michael Brustein has conducted a training session with Adult Education
providers. Additional training sessions will be conducted this fiscal year.
 

 

 

 




 
Final Report
ED-OIG/A06J0001                                     Page 42 of 43




            Enclosure 3: Arkansas E&E Calculation




 
         Final Report
         ED-OIG/A06J0001                                                                                                               Page 43 of 43

         2008-2009 EFFECTIVE and EFFICIENT CALCULATION:                                                            SAMPLE PROGRAM X
         Programs receiving adult education funds are required to meet or exceed 80% of the negotiated core indicators of
         performance.
         The E&E calculation uses "weighting" to give fair weight to each student with a goal (e.g., improve
         basic literacy skills and follow-up goals). Core Indicators 1.1-1.12 are based on programs' annual
         National Reporting System (NRS) Table 4 as reported in the AERIS student database. Core Indicator 4 is
         based on NRS Table 5 as reported in the AERIS student database.
         Core Indicators 2 3 and 5 completions are based on data matching with Department of Workforce Services
                               :




              and Department of Higher Education plus the addition of manually entered outcomes in AERIS.
         The Percentage (%) Completions is divided by the corresponding Negotiated Core Indicator to
         determine the Unweighted E&E Percentage. The Unweighted E&E Percentage is multiplied by the
         number of Enrolled Students in each Core Indicator to obtain an E&E Weighting.
         The E&E weightings are added up and then divided by the Total Number of Enrolled Students' Goals to
         determine the E&E RESULT.
             Enrolled Students = AERIS Table 4.1, Column B, and Table 5.1, Column D
             Completed Goals = AERIS Table 4.1, Column D and Table 5.1, Column F
             % Completions (calculated automatically) = AERIS Table 4.1, Column H; and Table 5.1, Column G
                                                                                                                   2008-2009
                                                                                                                   U nwe igh ted
                   CORE INDICATORS                                  Enrolled       Completed                         Negotiated       E&E%
                 EFL's and Follow-Up Goals)          GLE/SPL* Students               Goals          % Completion   Core Indicators Compl/Negot.) E&E Weighting
         ABE Beginning Literacy                1.1          0-1.9             17               7        41%             36%               114.4%           19.44
         ABE Beginning Basic                   1.2        2.0-3.9             66              25        38%             38%                 9 9 .7%        65.79
         ABE Intermed. Low                     1.3        4.0-5.9           411               212       52%             39%               132.3%          543.59
         ABE Intermed. High                    1.4        6.0-8.9           619               137       22%             40%                55.3%          342.50
         ASE Low                               1.5       9.0-10.9           390                77       20%             42%                47.0%          183.33
         ESL Beginning Literacy               ^1.7       SPL 0-1              31              12        39%             26%               148.9%           46.15
         ESL Low Beginning                     1.8         SPL 2              73              26        36%             33%               107.9%           78.79
         ESL High Beginning                    1.9         SPL 3              23               8        35%             35%                99.4%           22.86
         ESL Low Intermed.                    1.10         SPL 4              16              14        88%             34%               257.4%           41.18
         ESL High Intermed.                   1.11         SPL 5              11                4       36%             35%               103.9%           11.43
         ESL Advanced                         1.12         SPL 6               7               2        29%             32%                89.3%            6.25
         Get a Job                               2                          148                86       58%             58%               100.2%          148.28
         Retain a Job                           3                           325            28o          86%             75%               114.9%          373.33
         Get a GED                              4                           275               234       85%             64%               133.0%          365.63
         Enter Post-Secondary                   5                           55,                25       45%             39%               116.6%           64.10

                 TOTAL # of Enrolled Students' GOALS                 2467                 1149                                                             2312.65


              
                                                                                     E&E Results
                                   EFL = Educational Functioning Levels              "GLE = Grade Level Equivalents           "SPL = Student
         Proficiency Levels
         (ESL)^ EFL 1.6 is not included in E&E weighting because 1.6 (ASE High) is not a negotiated core indicator with the US Dept. of
         Education. See below.
         ASE High                         1.6        11.0-12.9          170             103               61%              N/A                 N/A
Arkansas Department of Career Education
Adult Education Division
         Total Students in Program                                     1834             627