oversight

Audit of the Massachusetts Rehabilitation Commission's Provision of Training Services and Client Monitoring for the period October 1, 1994 through September 30, 1995.

Published by the Department of Education, Office of Inspector General on 1998-06-30.

Below is a raw (and likely hideous) rendition of the original report. (PDF)

                       U.S. DEPARTMENT OF EDUCATION
                            OFFICE OF INSPECTOR GENERAL
                                 75 Park Place, 12th Floor
                                New York, New York 10007

                                          June 30, 1998

MEMORANDUM
TO:            John P. Higgins, Inspector General (Acting); Steven A. McNamara, Deputy
               Inspector General (Acting); Robert G. Seabrooks, Assistant Inspector General for
               Audit (Acting); Dianne G. Van Riper, Assistant Inspector General for
               Investigations; Hugh M. Monaghan, Jr., Assistant Inspector General for
               Operations; Richard T. Rasa, Director, Advisory and Assistance for State and
               Local Programs; and Regional Inspectors General for Audit

FROM:          Guido G. Piacesi
               Regional Inspector General for Audit, Northeast

SUBJECT:       FINAL AUDIT REPORT
               Audit of the Massachusetts Rehabilitation Commission=s Provision of
               Training Services and Client Monitoring for the period October 1, 1994
               through September 30, 1995

Attached is the above subject audit report. We are furnishing this report to you because it may
contain information of interest to you.

If you have any questions concerning the audit report, please contact me at (212) 637-6271.


Attachment
  Audit of the Massachusetts Rehabilitation Commission=s
   Provision of Training Services and Client Monitoring
for the period October 1, 1994 through September 30, 1995
                  Boston, Massachusetts




               FINAL AUDIT REPORT




               Audit Control Number A0160003
                          June 1998
                               NOTICE
 Statements that management practices need improvement, as well as other
conclusions and recommendations in this report, represent the opinions of the
 Office of Inspector General. Determination of corrective action to be taken
 will be made by appropriate Department of Education officials. This report
    may be released to members of the press and general public under the
                        Freedom of Information Act.
                                                     Table of Contents

Executive Summary ..............................................................................................................1

Audit Results ........................................................................................................................3

          Finding No. 1 Client Monitoring Needs Improvement .................................................3

                                                     Recommendations .........................................................4
                                                     Auditee Response ..........................................................4
                                                     OIG Reply ....................................................................5

          Finding No. 2 Counselors Did Not Always Consider Prior Education
                                and Skills When Establishing Vocational Goals .......................5

                                                     Recommendations .........................................................6
                                                     Auditee Response ..........................................................6
                                                     OIG Reply ....................................................................6

          Finding No. 3 Vocational Goal Changes Were Not Always Justified ...........................7

                                                     Recommendation ..........................................................8
                                                     Auditee Response ..........................................................8
                                                     OIG Reply ....................................................................8

          Other Matters Erroneous Case Service Reporting .......................................................8

                                                     Recommendation ..........................................................9
                                                     Auditee Response ..........................................................9
                                                     OIG Reply ....................................................................9

Background ........................................................................................................................10

Audit Objectives .................................................................................................................10

Methodology and Scope ......................................................................................................11
The Massachusetts Rehabilitation Commission Needs to Improve Procedures
for Monitoring Client Progress and Establishing Client Vocational Goals


Executive Summary   We have conducted an audit of the Massachusetts Rehabilitation
                    Commission=s (MRC) provision of services to clients receiving
                    training under the Vocational Rehabilitation program during the
                    period October 1, 1994 through September 30, 1995. Our audit
                    objectives were to determine whether MRC was (1) adequately
                    monitoring client progress and (2) considering prior client education
                    and vocational skills when establishing vocational goals. We found
                    that MRC case files did not contain evidence of sufficient annual and
                    periodic reviews to demonstrate adequate monitoring of client
                    progress; and that MRC did not always consider prior education or
                    vocational skills when establishing the client=s vocational goal.
                    Our review of 50 randomly selected client case files disclosed that:
                    !      33 of the 50 files (66 percent) did not contain evidence of all
                           required periodic and/or annual reviews;
                    !      10 of 14 case files of client=s with substance abuse problems
                           (71 percent) did not contain sufficient evidence of attendance
                           in recovery programs, as prescribed in their Individualized
                           Written Rehabilitation Program (IWRP).
                    !      10 of 37 case files containing evidence of previous client
                           education and/or vocational skills (27 percent) lacked
                           evidence that the education and/or skills were considered in
                           establishing vocational goals; and
                    !      18 of 20 case files containing evidence of one or more
                           changes to the client=s initial vocational goal (90 percent) did
                           not contain justification for the changes.
                    Improved monitoring of client progress can increase successful
                    rehabilitations. Similarly, utilization of prior education and/or
annual and periodic reviews and attendance of substance abusers in
recovery programs prescribed in their IWRPs, (2) consideration of
previous education and vocational skills when
Establishing client vocational goals; (3) documented justification for
goals which are inconsistent with existing education or vocational
skills; and (4) documented justification for changes in vocational
goals.
MRC generally agreed to implement procedures and training to
ensure improved documentation of annual and periodic reviews, as
well as conducting more frequent internal case file reviews. MRC
generally disagreed with our recommendations for improved
monitoring of substance abusers= attendance in recovery programs,
indicating our recommendation is contrary to the autonomous nature
of the IWRP. Similarly, MRC disagreed with our findings and
recommendations regarding utilization of clients= prior education and
vocational skills in obtaining gainful employment, and justification
for vocational goal changes; indicating the findings and
recommendations are not congruous with the letter and intent of
Policy Directive (PD) 97-04.

We disagree with the premises upon which MRC bases its objections
to the audit findings and/or recommendations. The findings and
recommendations for monitoring attendance of substance abusers in
recovery programs are, in fact, based on the client and counselor
responsibilities found in the IWRPs of the client case files reviewed.
The referenced PD is entirely congruous with and, in fact, supports
the findings regarding utilizing client education and skills (resources,
abilities, and capabilities). A synopsis of MRC=s response and our
reply is included after each finding of this report. MRC=s response,
in its entirety, is also included as an appendix to this report.
Final Audit Report - Massachusetts Rehabilitation Commission               Page 3                        ACN: A0160003



                                             AUDIT RESULTS

Finding No. 1                 MRC client case files did not contain evidence of sufficient periodic
                              and annual reviews to demonstrate adequate monitoring of client
Client Monitoring             progress. MRC officials indicated the lack of required monitoring
Needs Improvement             resulted from a lack of resources and heavy counselor case loads.
                              Our analysis of 50 randomly selected case files disclosed a
                              substantially higher success rate for clients whose case files
                              contained all required reviews.
                     C        Federal regulation 34 CFR 361.40 requires that each individualized
                     r        written rehabilitation program must be reviewed as often as
                     i        necessary, but at least on an annual basis. 34 CFR 361.41 requires
                     t        a procedure and schedule for periodic review and evaluation of
                     e        progress toward achieving rehabilitation objectives, and a record of
                     r        these reviews and evaluations.1
                     i
                     a
                              We randomly selected 50 client case histories from the universe of
                              1,391 MRC clients receiving training services during the Federal
                              fiscal year ended September 30, 1995. Inadequate monitoring of
                              client progress was found in 33 of the 50 case histories reviewed.
                              The following chart illustrates the number of case files missing one
                              or more annual and/or periodic reviews.

                                                   MRC Monitoring Results
                                                   2 (4.0%)

                                                              11 (22.0%)
                                                                               Missing Annual Review
                                20 (40.0%)
                                                                               Missing Periodic Review
                                                                               Missing Both Reviews
                                                                               No Exceptions Noted

                                                        17 (34.0%)
Final Audit Report - Massachusetts Rehabilitation Commission      Page 4                          ACN: A0160003



                              As illustrated, 20 of the 50 case files (40 percent) were missing both
                              annual and periodic reviews; 11 case files (22 percent) were missing
                              annual reviews; and 2 case files (4 percent) were missing periodic
                              reviews. Only 17 case files (34 percent) had all required reviews.
Monitoring                    MRC must also increase their monitoring of attendance for substance
Recovery of                   abusers in recovery programs. Included in the 50 case histories were
Substance                     14 clients with substance abuse problems. Our analysis disclosed
Abusers                       that 10 of the 14 client case files (71 percent) did not contain
                              sufficient evidence of continuous attendance in recovery
                              programs, as prescribed in the Individualized Written Rehabilitation
                              Program (IWRP). Interviews with MRC counselors also indicated a
                              need for closer monitoring of clients with substance abuse problems,
                              especially during their first year of recovery.

Monitoring                    Monitoring and case closure data suggest that monitoring may
May Impact                    impact successful rehabilitation. Of the 50 randomly selected cases,
Successful                    19 cases were closed successfully, 12 were closed unsuccessfully,
Rehabilitati                  and 19 remained open. Our analysis disclosed that 9 of the 19
on                            successfully closed cases (47 percent), contained all required
                              reviews, while only 2 of the 12 unsuccessful cases (17 percent)
                              contained all required reviews.

Heavy                         MRC officials acknowledged that monitoring clients on a regular
Caseloads                     basis plays an important part in the rehabilitation process. However,
Contribute                    MRC officials indicated that heavy caseloads prevent counselors
to the Lack                   from providing timely services, adequately following up on active
of Reviews                    cases, and documenting all actions affecting the client.

MRC                           At the time of our audit, MRC=s most recent internal Statewide Case
Identified a                  Review (1991) identified a decline in progress evaluations. MRC
Decline in                    believes the inability to hire additional program evaluators to perform
Monitoring                    internal reviews contributes to the decline in monitoring.
Services in
1991.
Final Audit Report - Massachusetts Rehabilitation Commission       Page 5                         ACN: A0160003



                                                   programs for substance abusers, and

                                           !       more frequent Statewide internal case
                                                   file reviews.
MRC Response                  MRC agrees with the finding and recommendations regarding
                              improved documentation of annual and periodic reviews, as well as
                              more frequent internal case file reviews; but disagrees with regard to
                              monitoring the attendance of substance abusers in recovery
                              programs. MRC states that our conclusions appear to be based
                              primarily on hypothesis and anecdote rather than an established,
                              clinically proven practice. MRC argues that the responsibilities of
                              the client and counselor are set forth in the IWRP and the audit does
                              not justify the imposition of a uniform monitoring regimen on all
                              clients with a diagnosis of substance abuse.
OIG Reply                     MRC=s implementation of procedures and training to ensure
                              documented periodic and annual reviews and more frequent internal
                              case file reviews should be sufficient. However, our conclusions
                              regarding the monitoring of attendance of substance abusers in
                              recovery programs are neither anecdotal nor hypothetical. The
                              finding presents a general failure by MRC to monitor attendance of
                              substance abusers in recovery programs, as prescribed in their
                              respective IWRPs. For example, the AClient and Counselor
                              Responsibilities@ section of the IWRP, for the first exception noted,
                              states A[Client] to attend AA on a basis she feels is appropriate to
                              her. [Counselor] to monitor.@ The case file contains no evidence of
                              client attendance in AA (Alcoholics Anonymous), or counselor
                              monitoring. Among the duties listed in the Massachusetts
                              Department of Personnel Administration=s Classification
                              Specification for Vocational Rehabilitation Counselors is: AMonitors
                              and evaluates client progress . . . to . . . determine whether or not
                              services, programs, or placements are meeting the client needs.@


Finding No. 2                 MRC did not always consider prior education or vocational skills
Final Audit Report - Massachusetts Rehabilitation Commission        Page 6                       ACN: A0160003



                              individual. The thorough diagnostic study includes in all cases, to
                              the degree needed, an appraisal of the individual=s employability,
                              personality, intelligence level, educational achievement, work
                              experience, personal, vocational, and social adjustment,
                              employment opportunities, and other pertinent data helpful in
                              determining the nature and scope of services needed.2
                              The failure to consider existing skills and/or education for the
                              purpose of obtaining gainful employment represents a potential
                              waste of Federal and State funds. For example, one client who had
                              previously worked in the food service field, established a vocational
                              goal in Printing/Photography, and attended school with assistance
                              from MRC. Subsequently, the client changed her goal to Social
                              Worker, again attending school with assistance from MRC. After
                              four years of training (1989 - 1993) for these two goals, the client
                              changed her goal to Culinary Arts, graduated from a Culinary Arts
                              school, and obtained employment in this field. The MRC counselor
                              supported the goal change based on the client=s prior work
                              experience. Had Culinary Arts been considered for the client=s initial
                              vocational goal, substantial resources could have been saved.
Recommendations               We recommend that the Assistant Secretary for OSERS require
                              MRC to implement procedures and training to ensure:

                                           !       utilization of prior education or
                                                   vocational skills in obtaining gainful
                                                   employment, except where a disability
                                                   precludes utilization of such education
                                                   or vocational skill; and

                                           !       documented justification for the
                                                   establishment of vocational goals
                                                   which are not consistent with a client=s
                                                   prior education and vocational skills.
MRC Response                  MRC states the audit may support Aa lack of documentation of
Final Audit Report - Massachusetts Rehabilitation Commission        Page 7                           ACN: A0160003



OIG Reply                     MRC=s Client Services Manual, states that the Individualized
                              Written Rehabilitation Program is intended to formalize the case
                              planning and management practice of establishing goals and
                              objectives and the means and time frames for their achievement.
                              Without documentation, the suggested use of testimonial evidence
                              regarding what may have been considered several years ago is
                              unreasonable. The example cited in the finding is precisely the type
                              of condition to which the finding and recommendation is directed.
                              The finding is entirely congruous with, and in fact supported by, the
                              referenced Policy Directive, PD 97-04, which states AThe
                              employment goal for an individual . . . must be based, primarily, on
                              the individual=s strengths, resources, priorities, concerns, abilities and
                              capabilities.@ Accordingly, we maintain the recommendations
                              should be implemented by MRC.


Finding No. 3                 MRC client case files did not always contain justification for changes
                              to client vocational goals. Vocational goal changes were found in 20
Vocational Goal               of the 50 randomly selected client case files. Justification for the
Changes Were Not              changes was not found for 18 (90 percent) of the 20 clients. MRC
Always Justified              officials are aware that vocational goal changes require proper
                              justification by the counselors under current State regulations. They
                              believe the demands on counselors to produce more rehabilitated
                              clients, and the difficulty of working with clients who have less
                              educational and vocational experience may be some of the reasons
                              for changing vocational goals.
Criteria                      MRC Client Services Manual, Chapter 107 CMR G11.06 states that
                              form CS-17e is used to record any amendment to the original IWRP,
                              which could be a different vocational goal, changes in services, or a
                              program closure. When form CS-17e is used for a new vocational
                              goal, the counselor must record the occupation and a three or more
                              digit code from the Dictionary of Occupational Titles and a
                              narrative justification.
Final Audit Report - Massachusetts Rehabilitation Commission        Page 8                            ACN: A0160003



                              Records Technician. A third change of goal occurred in June 1995.
                              The client attended school for one week and then changed his mind
                              because he was the only male in the program.
                              The vague rationale provided by the client for the above changes do
                              not justify those changes. In accordance with 34 CFR 361.33, cited
                              on page 5 of this report, a thorough diagnostic study, including an
                              evaluation of employment opportunities and the client=s ability to
                              acquire the vocational skill, is required to determine the nature and
                              scope of services to be provided. Without supporting evidence to
                              the contrary, it is assumed that employment opportunities exist and
                              the client has the ability to acquire the skills for the vocational goal.
Recommendation                We recommend that the Assistant Secretary for OSERS require
                              MRC to implement procedures and training to ensure narrative
                              justification for all vocational goal changes, as prescribed by State
                              regulations. The new procedures should require convincing
                              documented support when the justification contradicts the initial
                              thorough diagnostic study.
MRC Response                  MRC reiterates the finding is in conflict with PD 97-04. MRC again
                              states the example serves no useful purpose and the finding Asuggests
                              a degree of counselor control over the planning process that is not
                              supported by the RSA policy directive.@ MRC believes that Ano
                              recommendation, beyond improved documentation of the
                              justification for case activity, be made.@
OIG Reply                     The finding is not in conflict with the referenced PD. As stated in
                              the PD, the purpose of Title I of the Rehabilitation Act of 1973, as
                              amended (the Act) is Ato assist states in operating a comprehensive
                              . . . and accountable program of vocational rehabilitation . . .@ We
                              do not believe the PD inhibits a Designated State Unit=s ability to
                              prevent numerous unjustified vocational goal changes such as those
                              demonstrated by the example cited in the report. Otherwise, control
                              over the efficiency, integrity, and accountability for program
                              expenditures would not exist. Accordingly, we maintain the
Final Audit Report - Massachusetts Rehabilitation Commission       Page 9                          ACN: A0160003



                              Based on information and supporting documentation provided by
                              MRC, we concluded that the reporting of training services for five of
                              the eight clients resulted from transcription or data entry errors.
                              Four cases were coded to incorrect client numbers and one pertained
                              to a diagnostic evaluation improperly coded as training. In each of
                              the three remaining cases, the courses were used as a diagnostic tool,
                              part of an extended evaluation of the clients= potential to benefit from
                              vocational rehabilitation services.
Recommendation                We recommend that the Assistant Secretary for OSERS require
                              MRC to review and correct all improperly coded Case Service
                              Reports from 1994 through the most recently submitted report and
                              provide training to all counselors and staff responsible for data input
                              to preclude future erroneous coding of case service data.
MRC Response                  MRC does not agree that a coding error occurred regarding the
                              provision of Postsecondary education to applicants, as a means of
                              evaluating readiness for service. MRC states: AIn fact, the use of
                              Postsecondary training for evaluation is suggested and endorsed by
                              RSA.@
OIG Reply                     The audit report does not take exception to the use of Postsecondary
                              training for the purpose of evaluation. The report initially took
                              exception only to the reporting for all eight clients. However, in
                              their response to our draft report, program officials informed us of
                              an apparent ambiguity in the Reporting Manual for the Case Service
                              Report, pertaining to reporting training services. Accordingly, we
                              concur that three clients who received training during extended
                              evaluation may not warrant an exception. However, other reporting
                              errors were found in five of the eight cases. In response to our
                              request for an explanation of the reporting for the eight clients, MRC
                              responded that Aonly 3 actually received any training . . . @, 1
                              received no training but was incorrectly coded as having been
                              >provided= same@, and for the remaining four, A . . . in every case
                              these incorrect charges were made as a result of transcription or data
                              entry error.@ Except for reducing the number of exceptions noted,
          Final Audit Report - Massachusetts Rehabilitation Commission     Page 10                   ACN: A0160003




Background                 The purpose of the Vocational Rehabilitation Services program
                           (Title I) is to assist States in operating a program which provides
                           vocational rehabilitation services for individuals with disabilities so
                           that such individuals may prepare for and engage in gainful
                           employment. In order to receive Vocational Rehabilitation Services
                           funds, a State must submit a three-year State plan, designating the
                           State agency responsible for administering the plan and providing for
                           financial participation by the State.

                           States and the Federal government share in the total cost of the
                           program, with the States providing at least 20 percent of the total
                           cost of the program. The Massachusetts Rehabilitation Commission
                           (MRC) is the designated state agency for the Commonwealth of
                           Massachusetts. MRC=s administrative offices are located in Boston,
                           Massachusetts. District and area offices are located throughout the
                           State. Federal and State funding, for the fiscal year ended
                           September 30, 1995, were $33.5 million and $15.4 million,
                           respectively.

                           Recent audits, conducted by our office, of the Vocational
                           Rehabilitation State Grant Program disclosed that frequent contact
                           with vocational rehabilitation clients seems to improve successful
                           rehabilitation. Also, vocational rehabilitation staff report that in
                           order to improve the employment opportunities of persons with
                           disabilities, some form of restriction on client choice of vocational
                           goals should be provided.


Audit Objectives           The objectives of our audit were to determine whether:

                           !       adequate monitoring is performed to provide reasonable
                                   assurance of client progress, and
          Final Audit Report - Massachusetts Rehabilitation Commission      Page 11                  ACN: A0160003




                           !       Case Service Report data (RSA 911) for Federal fiscal years
                                   1990 through 1994; and

                           !       Statewide Single Audit working papers.

                           We interviewed officials from the Rehabilitation Services
                           Administration, State Vocational Rehabilitation Agency (MRC) and
                           MRC Vocational Rehabilitation Counselors. We also reviewed 50
                           randomly selected case files from the universe of 1,391 clients
                           receiving training services through MRC during fiscal year 1995.

       Data Reliability    We did not rely extensively on computer processed data. All audit
       Assessment          tests were performed using source documentation. However, our
                           review of data submitted to ED by MRC, via data tape, for the Case
                           Service Report (RSA 911) disclosed erroneous data, the extent of
                           which is fully described in the AOther Matters@ section of this report.

                           Except for the above noted review of case service data, our audit
       Audit Period        covered the period October 1, 1994 through September 30, 1995.
                           Our review of case service data covered the period October 1, 1989
                           through September 30, 1994. Our field work was conducted at
                           MRC=s central offices between May 9, 1996 and October 30, 1996.
                           Audit work and ongoing resolution of audit exceptions continued at
                           our offices through the date of our exit conference on October 22,
                           1997.

                           Our audit was conducted in accordance with government auditing
                           standards appropriate to the scope of the review described above.


Internal Controls          As part of our audit we assessed the system of internal administrative
                           controls, policies, procedures, and practices applicable to the
                           objectives of our audit. Our assessment was performed to determine
            Final Audit Report - Massachusetts Rehabilitation Commission     Page 12                 ACN: A0160003



                             material weaknesses in the internal controls. However, our
                             assessment disclosed internal control weaknesses which could
                             adversely affect the efficiency and effectiveness of MRC=s
                             administration of the vocational rehabilitation program. These
                             weaknesses included a lack of adequate internal oversight, training,
                             and enforcement to ensure compliance with Federal and State
                             requirements pertaining to all of the above noted control categories,
                             the effect of which is fully disclosed in the AUDITS RESULTS
                             section of this report.
MRC Response                 MRC does not agree that the findings of the audit report disclosed
                             internal control weaknesses which could adversely affect the
                             efficiency or effectiveness of the agency=s administration of the
                             program. MRC contends that limiting client choice would more
                             likely result in increased conflict and reduced efficiency and
                             effectiveness.
OIG Reply                    The reported control weaknesses which could adversely affect the
                             efficiency and effectiveness of MRC=s administration of the
                             vocational rehabilitation program, are further evidenced by MRC=s
                             response. MRC=s presumed lack of control over the establishment
                             and subsequent changes to client vocational goals demonstrates the
                             seriousness of the control weaknesses reported. If uncorrected,
                             abuse of client choice and respective State and Federal resources, as
                             noted in the examples cited in this report, will continue.
                                          REPORT DISTRIBUTION LIST
                                          AUDIT CONTROL NO. A0160003


                                                                                                                               No. of
Auditee                                                                                                                        Copies

Mr. Elmer C. Bartels, Commissioner ...........................................................................................1
Massachusetts Rehabilitation Commission
Fort Point Place
27-43 Wormwood Street
Boston, Massachusetts 02210-1616


Action Official

Judith E. Heumann .....................................................................................................................4
Assistant Secretary
Office of Special Education and Rehabilitative Services
Room 3006, Mary E. Switzer Building
330 C Street, SW
Washington, D.C. 20202


Other ED Offices

Commissioner, Rehabilitation Services Administration ................................................................1
Regional Commissioner for Rehabilitation Services .....................................................................1
Office of Public Affairs................................................................................................................1


Office of Inspector General

Inspector General (Acting) .........................................................................................................1
Deputy Inspector General (Acting) .............................................................................................1