United States General Accounting Office “GAO Report to the Committee on Armed Services, House of Representatives March 1990 DOD OVERSEAS SCHOOLS Additional Assurances of Educational Quality i:7 Needed G-0 i$f%%&iiI”;iF .. Human Resources Division B-235288 March 15,199O The Honorable Beverly B. Byron Chairman, Subcommittee on Military Personnel and Compensation Committee on Armed Services House of Representatives Dear Madam Chairman: This report responds to the House Armed Services Committee’s request-included in its report, which accompanied the fiscal year 1989 National Defense Authorization Act-that we review schools operated by the Department of Defense (DOD) for the dependents of military and WD civili;tn personnel located overseas. The major issues discussed in the report are the adequacy of information on the quality of education provided by the schools and the responsiveness of the school system to parental concerns. We are recommending that the school system (1) use measures in addition to test scores to assess education quality; (2) maintain better evidence that teachers are qualified and waivers of high school graduation requirements are proper; and (3) ensure that school advisory committees, which are comprised of parents and teachers, have the opportunity to advise principals on school policy issues. We are sending copies of this report to the Secretary of Defense; the Director, Office of Management and Budget; appropriate congressional committees; and other interested parties. If you have questions concerning this report,, please call me on (202) 275-1793. Other major contributors to this report are listed in appendix IV. Sincerely yours, Franklin Frazier Director, Education and Employment Issues Executive Summary quality and should be supplemented with other indicators-such as pro- motion rates and measures of the variety of course offerings-for a more comprehensive assessment of the quality of the schools. This is particularly relevant for the DOD system because its students are highly mobile, often attending these schools for only a few years, and their test scores may reflect education received elsewhere. (See ch. 2.) DODalso needs better procedures for documenting that the schools have quality teachers and that students meet graduation standards. Files fre- quently lacked the required documentation that (1) teachers met, mini- mum employment requirements, and (2) high-school students were properly granted exceptions when permitted to graduate without mcet- ing minimum course requirements. (See ch. 2.) While school advisory c*ommittees have been established to provide par- ents and teachers with a forum for expressing their views on school operations, they seldom exercised their specific authority to advise school principals on budgets and course curricula. Some parent members believed that they lacked sufficient influence in committee meetings to direct attention to thr,sc! matters and were unaware that they could elc- va(e concerns that a~‘()unrc>solvcd at the school level to DOI)manage- ment. (Set ch. 3.) LXIL)has implemented widely used drug and alcohol abuse programs in its schools and has generally corrected facilities’ shortcomings, such as inadequate space and Itbaky roofs, which were identified by its accredit- ing organization. (See VI-I.4.) Principal Findings DOD Students Score Above IWI) students have c*onsistcntly scored above average on nationally rec- Average on Standardized ognized standardized ac*hievement and aptitude tests. For example, dur- ing the 1987-88 school yea]‘, DOD students took t.he Comprehensive Test Tests of Basic Skills and on average scored above the 50th percentile for all sub,jects. Similarly. ov(‘r the past 4 years, DOD’Sstudents exceeded the national average on all sub,jccts on the American College Testing exam and on the verbal part of the Scholastic Aptitude Test. Also the students lucre near or slightly abovcl the national average on the mathematics piW of the Scholastic .Ipt il ude Test. (See pp. 12-l 6.) Page d GAO/HRD-90.13 DOD 0wrsras Schools . while there are equal numbers of parents and teachers on the commit- tees, the meetings arc often attended by nonvoting school administra- tors and teacher union representatives who many parents believed strongly influence the members. (See pp. 21-23.) Drug Prevention Programs DODhas implemented drug and alcohol abuse prevention programs for elementary and secondary students in all of its schools. These programs Established are widely used in I1.S. school systems to help reduce student drug and alcohol abuse. (See pp. 2526.) Most Facilities Problems School facilities problems, such as inadequate space and emergency lighting, leaky roofs, and unattractive landscaping, have been cited in Corrected accreditation survey rcaports by non’s independent school accrediting association. GM’s rcvit%w of the most recent accrediting reports and its inspection of the facilities at 30 schools with problems identified by the accrediting association indicated that (1) the association did not con- sider most of thra probklms serious enough to detract from the quality of education and (2) INHI had cx)rrccted over 70 percent of the reported problems. (See pp. 2627. I To provide parents and school system management additional indicators Recommendations that, their schools are providing students with high-quality education, (;.A()recommends that I xx): l!se, in addition to tchst scores, other measures to assess education qual- ity. (See p. 20.) . Ensure adequate documentation is maintained for (1) teachers’ qualifi- cations, and (2) the basis for granting waivers of high-school graduation requirements. (See p. 20.) . Ensure that advisory committees are provided the opportunity to review and advise school principals on school policy issues, specifically curricula and budget issues, by requiring the committees to document that they have been given that opportunity, and are aware that they can elevate unresolved c’onccrns to school system management above the principal level. (SW p. 23.) DOD agreed with GAOk recommendations. (See app. III.) Agency Comments Page 5 GAO/HRD-90.13 DOD Oversras Schools Tables Table 2.1: Documentation of Teacher Employment 18 Qualifications in Personnel Files Table 2.2: Teachers Lacking Required Certifications, and 19 Resulting Action (Germany and Pacific Regions) Table 4.1: Schools With Facilities Problems Identified by 27 NC4 Table 4.2: Facilities Problems Identified by NCA 27 Table II. 1: DOD High-School Graduates at Schools in 29 GAO’s Analysis Table 11.2: Teacher 1Universes 30 Figures Figure 1.1: DOD Dependents Schools Organizational 9 Structure Figure 2.1: DOD Students in Grades 1 - 6 Score Well on 13 Achievement Tests (School Year 1987-88) Figure 2.2: DOD Students in Grades 7 - 11 Score Well on 14 Achievement Tests (School Year 1987-88) Figure 2.3: DOD Students’ SAT Scores Are at or Above 15 Average (School Years 1985-88) Figure 2.4: DOD Students’ ACT Scores Are Above Average 16 (School Year 1987-88) Abbreviations ACT American College Testing Exam CTl3S Comprehensive Test of Basic Skills DARE Drug Abuse Resistance Education DOD Department of Defense GAO General Accounting Office NCA North Central Association SAT Scholastic, Aptitude Test Page 7 GAO/HRD-9043 DOD Overseas Schools Chapter 1 Introduction Figure 1.1: DOD Dependents Schools Organizational Structure Offlce of Dependents Alexandria. Virginia \ Germany Panama Atlantic Region Mediterranean Paclflc Region Region Flegkm E.%tCOte. Region Okinawa, Japan Albrook AFS, Wiesbaden, England Madrid. Spain Panama West Germany \ 4 Districts 8 Districts 3 Districts 4 Dlstrlcts The DODschool system has over 13,500 employees, 9,800 of whom arc teachers. The system performs most of its own budget, supply, and teacher recruitment activities, although it relies on the military depart- ments for some logist ical. financial, and personnel support on a reim- bursable basis. DODuses the North Central Association of Colleges and Schools (~4) to evaluate and accredit I he educational quality of its schools. The XC:\, founded in 1895, is that largest and oldest of the six regional accrediting associations. It accredits annually over 6,000 elementary schools, mid- dle/junior high schools, high schools, and institutions of higher educa- tion in 19 states and overseas. The 1978 act requirt>s noI) to establish school and installation advisory committees to pro\id~~ communication links between t,he school systems and the communititxs they serve. Each school’s advisory committee Page 9 GAQ/HRD-W-13 DOD Overseas Schools Chapter 2 Additional Assurances of Educational Quality Needed DOD needs to provide its management and parents with more assurances that it is providing high-quality education to its students. Although LX)II arranges for periodic evaluations of its schools by an independent accrediting agency and reports students’ performance on standardized achievement and aptitude tests, additional measures of school effective- ness-such as promotion rates and the variety of courses students take-are needed to better assess educational quality. Such additional measures are particularly important for the DOD system because the standardized test scores of its students-who tend to be highly mobile-likely reflect the education they received in other school systems. DOD also needs to better assure that its teachers are fully qualified and that high-school graduates meet graduation requirements. DOD assesses the quality of education it provides through (1) its school How DOD Assesses Its accreditation process, (2) a periodic curricula review and improvement Schools effort, and (3) standardized achievement and aptitude tests. School Accreditation DODcontracts with s( .Afor accreditation reviews of its schools. To be accredited, a school must meet NCA standards for its educational pro- gram, teacher qualifications, school facilities, school supplies, and administrative servrces. The primary objectives of the accreditation pro- cess are to (1) ensure t hat schools provide educational programs of high quality for all students, (2) encourage continuous appraisal and improvement of the school program, (3) foster public confidence. and (4) assist in identifying educat,ionally deficient schools, Schools are accred- ited by the association if they pass an on-site review every 5 years, The schools review and report on their operations annually and prepare a school improvement plan, based on an internal evaluation, before the review. As of school year 1987-88, NCZ had approved all DOD schools. .-. Curricula Review and DOD uses a 7-year curricula development cycle to keep current with the latest trends and ensure the appropriateness of its programs. The pro- Development cess uses educational specialists from the regional offices and headquar- ters who survey teachers regarding the effectiveness of each of their curricula, such as social studies, science, and mathematics. The special- ists spend the first 2 years of a cycle reviewing, selecting, and ordering instructional materials. Teachers begin using the new materials at the beginning of the thnd year. In the fourth and fifth years, the specialists ~~~ -- Chapter 2 Additional Assurances of Educational Quality Needed Figure 2.1: DOD Students in Grades l-6 Score Well on Achievement Tests (School Year 1987-88) 70 65 60 55 50 Grade 1 Grade 2 Grade 3 Grade 4 Grade 5 Grade 6 r----Y u Reading Language Arts Math Page 13 GAO/HRD90-13 DOD Overseas Schools Chapter 2 Additional Assurances of Educational Quality Nreded Figure 2.3: DOD Students’ SAT Scores Are at or Above Average (School Years 1985~88) 500 Score 490 480 470 460 450 440 430 420 410 400 1985 School Year - Math . Nation ---- Math-Dal - VerW - Nation ..s. Verbal-DOD Similarly, as shown in figure 2.4, for school year 1987-88, the average performance of DOI) students on ACT exceeded the national average in all subject areas. Page 15 GAO/HRD90-13 DOD Overseas Schools Chapter 2 Additional Assurances of Educational Quality Nerded requirements; and (4) participation of students in the arts and extracur- ricular activities. Similarly, in an August 1987 report,’ the Congressional Budget Office noted that test scores alone are an unreliable measure of the quality of an educational program because many educational, socie- tal, and other factors can influence the scores. According to DOD, it collects much of the data needed to report addi- tional indicators of educational quality, including records of each stu- dent’s attendance, academic progress, and grades. IIowever, this kind of information is not routinely included in its annual report to the Congress. r)ou often lacks documentation that student.s meet its minimum requirc- DOD Needs Better ments for graduation and t,hat teachers are fully qualified to teach their Procedures to grade levels and sul),jects. Hctter documentation could help L)OD’S man- agement and students‘ parents determine whether students are rcceiv- Document That ing a high-quality edllc,at,ion. Students Meet Graduation Requirements and Teachers Are Qualified Somcl Students Graduate DODrequires students to complete a minimum of 20 credits (16 in required subject areas and 6 electives) in order to graduate from high Wit hut Meeting Minimum school. However, DOI)permits school principals to waive certain require- Requirements ments-or substitute alternative courses-if they believe it would br in the student’s best interest. In such instances, school principals are required to maintain a record of the rationale for the waiver or substitution. We reviewed transcripts of all students who graduated from the Korea district in 1988, and samples of those who graduated from the larger Frankfurt and Philippines districts. All but 1 of 96 students WC reviewed in Frankfurt met minimum graduation requirements. On the other hand, 25 percent of those, in Korea and 10 percent of those in the Philippines did not meet the wyl~trements. For example, some students were Pagr 17 GAO/HRD-90.13 DOD Overseas Schools Chapter 2 Additional Assurances of Educational Quality Needed Most Teachers Are DOD requires its teachers to be certified to teach their grade levels and subjects. As part of its accreditation process, KCA assesses the creden- Properly Certified tials of all newly hired teachers. xx reviewed the files of 1,401 teachers in the Germany and Pacific regions for school year 198687, and 1,743 for school year 1987-88. As summarized in table 2.2, it found that few teachers (only about 1 per- cent) were teaching subjects for which they were uncertified. Subse- quently, all of these either made up their deficiencies and were certified, were reassigned to teach a subject for which they were certified, volun- tarily left the school system, or were removed from their positions. Table 2.2: Teachers Lacking Required Certifications, and Resulting Action Teacher Subsequently met (Germany and Paclflc Regtons) files Lacked requirements or School year reviewed certifications were reassigned Terminated 1986-87 1,401 16 9 7 1987-88 1 74.1 19 14 5 Our review of a sample of 225 teachers’ files showed that most con- tained evidence that teachers were certified. Certifications that teachers were qualified to teach were missing for: . 1 of the 60 files (less than 2 percent) in Korea. l 1 of the 100 files ( 1 percent) in Frankfurt. l None of the 65 files in the Philippines. DOD officials gave various reasons for the lack of evidence of certifica- tion, including that local shortages of teachers in certain subjects required hiring teachers without full certifications, new teacher arrivals sometimes did not have their certificates with them, and certificates were lost. Students who attend i)oD schools score well on standardized achieve- Conclusions ment and aptitude tests. However, these scores are only one measure of education quality and should be supplemented with other indicators to provide a better assessment of the schools. Also, although students who graduate without meeting minimum requirements may have been granted valid waivers by their principals and teachers may be fully qualified, DOD files often lacked required documentation. To provide for a more comprehensive assessment of the school system, DOD should (1) develop additional measures of education quality and (2) ensure that Page 19 GAO/HRD-90.13 DOD Overwas Schools Chapter 3 Advisory Committees Can Be More Effective - Advisory committees are required by law and are intended to provide parents and teachers a means for raising and resolving their concerns about school operations. DOD established advisory committees at each school and installation we visited. However, these committees have gen- erally focused on school support issues, such as transportation and school lunch programs. and have seldom exercised their authorit,y to advise school principals on curricula and budget matters. In addition, some parents believe that their influence on advisory committees is lim- ited by the requirement that there be equal numbers of parent and teacher members because nonvoting participants, particularly principals and teacher union representatives, often attend the meetings and influ- ence the positions taken by teachers on issues. Assurances are needed that advisory committees are provided the opportunity to review and report their views on the entire spectrum of school plans and operations. -~ The Defense Dependents’ Education Act of 1978 requires DOD to estab- Advisory Committee lish school advisory committees to provide advice to school principals Functions and installation commanders on school affairs. As specified in the law, these committees are composed of an equal number of parents whose students are enrolled in the schools and full-time professional school employees (usually teachers). They are authorized to provide advice and make recommendations on almost any school-related issue, specifically recommendations on curricula and budgets and on installation-provided support, such as transportation, maintenance, and school meals. Where there is more than one school on an installation, the latter function is performed by an “installation” advisory committee comprised of mem- bers from the school advisory committees. DOD procedures provide that when a committee is unable to resolve an issue with a principal or com- mander, it can put its concerns in writing and refer them to successively higher levels-up to the Director of the Office of Dependents Schools- until resolution is reached. .__ Advisory committees for the schools we visited have tended to focus on Committees’ Influence such school operat,ions issues as lunch programs and pedestrian safety, on Curricula and rather than on revit\ving and making recommendations on school curric- Budget Issues Limited ula and budgets. IGcause DOD is not required to document what issues advisory committees have considered, we could not determine how many committees wart given the opportunity to review and comment on thcsc matt,ers. Howc~\ er. available documentation showed that during Page 21 GAO/HRD-90.13 DOD Overseas Schools Chapter 3 Advisory Committees Can Be More Effective Legislation requires that school advisory committees include an equal Composition of School number of parents and professional school employees (usually teachers) Advisory Committees and a nonvoting teacher union representative. This requirement is intended to encourage family members and school employees to partici- pate in school operations, but some members believe it limits parental influence. Parent and teacher members from 8 of the 18 school advisory committees that we interviewed believed that these committees would be more effective if parent representation was increased. These mem- bers indicated that-even though the committee membership is com- prised of an equal number of parents and professional school employees-the parents often felt they had little influence at meetings because of the presence of nonvoting participants (such as school princi- pals and teacher union representatives) who they perceive can exert influence over the teacher members. Similarly, the Assistant Secretary of Defense for Force Management and Personnel indicated during the same October 5, 1988, hearing at which we testified, that some school community members believe that having an equal number of parents and professional school employees moves the focus of school advisory committees away from community issues to teachers’ concerns. The Assistant Secretary added that this has led to a commonly held notion that the school advisory committees are not fully responsive to parental concerns. Some advisory committee members believe they have little influence on Conclusions their schools. The members felt that school principals often limited their opportunity to discuss curricula and budget issues at their meetings, and many were unaware that they could elevate unresolved concerns and complaints to higher levels of management in the school system. Conse- quently, DOD should tnakc sure that advisory committees have been pro- vided the opportunity to comment on such issues as school curricula and budgets. Such assurances, combined with DOD'S current initiatives, should help to improve communications between the schools and the communities they strvc. We recommend that the Secretary of Defense require the Office of Recommendation Dependents Schools to ensure that advisory committees are provided the opportunity to review school policy issues and to advise school prin- cipals on them, specifically curricula and budget issues. This can be assured by requiring the committees to document that they have been Page 23 GAO/HRD-90.13 DOD Overseas Schools Chapter 4 DOD Has Been Responsiveto SubstanceAbuse and Facilities Problems In response to drug and alcohol use by its students, MID has implcmcnt~ctl substance abuse prevention programs that are used by other school sys- tems for elementary and secondary students in all its schools. I)(N) also is implementing a pilot drug abuse prevention program for fift,h and sist h grade students. In addition, DOL) corrects most problems with its physical faciliticls noted by its accrediting agenc’y. -.~ A 1987 drug and akohol survey, conducted by the lJniversit,y of Mich- Drug and Alcohol gan for the Kational lnst it utc of Drug Abuse, indicated that, about 53 Abuse Prevention percent of I)oL)‘s 1987 high-school seniors have used marijuana, c.ocaine. or other illicit drugs at some point in their lives. Among statesid<\ high- Programs Have Been school seniors, ac.cotdmg to the survey, 57 percent reported having used Implemented an illicit drug. The stutly also estimated that about 9,s percent 01’ MN) high-school seniors had usc>dakohol at least once in their lifetimc,s. uxw pared with about O:! ~x-rcx~nl of stateside seniors. ISec-auseof conc(lrns ;Iljont drug and alcohol use in its schools, I)OI) has implemented drug an(l al~)hol abuse prevention programs for all grades. One program, rc,ferrcxtl to as “Ilere’s Looking at You, 2000.” is for stu- dents in kindcrgart cbnthrough grade eight. The program provides infor- mation on drugs and ;~lcohol. gives opportunitic,s for peer teaching and parental involvcmc*nl and is designed to assist students in making responsible decisions. .Inothcr program, called “Together,” is used in grades 9 through 12. and is an alcohol and drug education program that includes developing shills in making decisions, managing stress, solving problems, and de\xslcq)in g a positive self image. The “Togcthcr” program also includes a con\potlcM for kindergarten through gradr ($$t calltxtl “Choosing for Yout5t,Il’.” The Pacific rrgion Itul ~Iemcnt.ed the two programs primarily by training sc*hool staff mcmbr\rs ;tnd t,caehing the programs in the classrooms. Dur- ing school year 1987-X8. the developers of “Here’s Looking at YOII, 2000” trained teams 01’ three to five teachers. counselors, nurses, and administrators from c~lr elementary and junior high school in the region. Thcsc teams t Ilcln returned to their schools, trained othtxr school cmployet~s, and UNI~~II~T~YI parent and community awareness prtscnta- tions. For the “‘l’ogc,l trtlr” program. consultants trained the region’s high-school health toach(~rs. counselors, nurses, and administrators in school year 198G 1!M;. Chapter 4 DOD Has Been Responsive to Substance Abuse and Facilities Problems Table 4.1: Schools With Facilities Problems Identified by NCA District Number of schools Korea Okinawa Philippines Frankfurt Totals With problems 5 7 IO 8 30 That had corrected all problems 1 3 a 1 13 That had some uncorrected oroblems 4 4 2 7 17 As shown in table 4.2, non corrected most of the 165 problems (about 70 percent) identified by its accrediting agency. Other than the deficiencies identified at the one school that did not meet standards, NC4 did not con- sider the remaining uncorrected problems serious enough to detract from the quality of education at the schools. by NCA District Number of problems Korea Okinawa Philippines Frankfurt Totals All schools 52 26 39 46 165 For schools that had corrected all problems 6 8 31 4 49 For schools wtth some uncorrected problems 46 20 8 42 116 Corrected 32 13 3 20 68 Uncorrected 14 7 5 22 48 School principals cited various reasons why some problems were uncor- rected at their schools. The reasons included plans to replace some facil- ities (which takes longer than making repairs), slow action on schools’ repair requests, and difficulty in identifying ways to fix the problems. The principals plan to continue their efforts to follow up on repair requests and to identify ways to correct the problems. Page 27 GAO/HRD-90-13 DOD Overseas Schools Appendix II Scopeand Methodology ~ We reviewed pertinent legislation and DOD documents and interviewed Office of Dependents Schools officials, including the director, deputy director, and their staffs. To address educational quality, we interviewed DOD and Department of Education officials and obtained and analyzed information, such as (1) the results of periodic evaluations of DOD schools by &CA,(2) annual cur- ricula evaluation plans, (3) standardized achievement and college apti- tude test scores, (4) parent attitude surveys, and (5) DOD assessments of educational quality. In addition, to see if non was enforcing its minimum high-school gradua- tion requirements, we reviewed 193 student transcripts randomly selected from students who graduated from six schools in the Frankfurt, Philippines, and Korea dist,ricts during the 1987-88 school year. To per- form our analyses, we designed a structured data collection instrument to ascertain, among other things, evidence of education, such as courses taken and credits earned. The following table identifies the district offices, student graduate universes in each district, and the sample size of the student transcripts reviewed. Table 11.1:DOD High-School Graduates at Schools in GAO’s Analysis Universe of graduated Germany region students Sample size Frankfurt dlstrlct 457 96 .~~ Pacific Region Korea dlstrlct 28 28 Phlllpplnes district 198 -69 Total 683 193 Since there were only 28 high-school students who graduated at the two schools in Korea, we reviewed all students’ transcripts and related docu- mentation. We randomly sampled the universe of graduated students from the Frankfurt and Philippines districts by using a computer-gener- ated list of random numbers. To determine if proper documentation was available on teacher qualifi- cations, we randomly selected 225 teachers’ files from the 934 classroom teachers at 33 schools in the Frankfurt, Korea, and Philippines districts. We then reviewed their personnel files at civilian personnel offices to determine whether the files were complete and provided an adequate basis for assuring that teachers are properly qualified to perform their duties. WC also designed a structured data collection instrument for use Page 29 GAO/HRD90.13 DOD Overseas Schools Appendix II Scope and Methodology To address the status of DOD'S drug and alcohol abuse prevention pro- grams, we interviewed program and fiscal officials at the headquarters and regional levels and reviewed available documents and reports describing DOD'S drug and alcohol prevention activities. To evaluate the physical condition of school facilities, we relied on the results of NCA’S most recent evaluation reports to determine the prob- lems attributed to school facilities and observe the conditions related to those problems. We visited 30 schools in the Frankfurt, Korea, Okinawa, and Philippines districts to follow up on the facilities problems NCA had identified in its reports. Page 31 GAO/HRD90-13 DOD Overseas Schools - Appendix ID Comments From the Department of Defense - r 2 asking their opinion on school effectiveness overall. Fifty-five percent or 75,426 parents responded. The survey results for the system have been published widely, and each school will receive its own results which will be used to develop school improvement plans. The need to build confidence among parents that they have a significant role in the DOD schools comes through clearly in the report. The Director of DoDDS has identified effective communication with parents as a major organizational goal. The concerns that emerge in the report are a reminder that the DOD efforts must be untiring as parents and other members of the community are informed about the schools, especially about how parents can be most effective in serving on School Advisory * Committees. A newly developed video tape and study guide for use in training Committee members received generally favorable evaluations at the end of the 1988-1989 school year. The fact that confusion still exists in the minds of some parents about how the Committees work, despite these training efforts, means that the DOD must and will do more to assist parents in participating in the governance of its schools. Detailed DOD comments on the findings and recommendations of this report are enclosed. The DOD appreciates the opportunity to comment on the draft report. Sincerelv. Enclosures: As Stated Page 33 GAO/HRD-90-13 DOD Overseas Schools Appendix III Comments From the Departmrnt of Defense L Student performance on standardized tests: The DOD students participate in the Comprehensive Test of Basic Skills, the Scholastic Attitude Test, and the American Collese Testino programs. - The GAO reported that, as of 1987-1988 school year, the North Central Association of Schools and Colleges had approved all the DOD schools. The GAO further reported that, during school year 1987-1988, DOD students scored higher than the 50th percentile in all curriculum areas at all grade levels, as measured by the Comprehensive Test of Basic Skills. The GAO noted that DOD students scored higher than the national average on the verbal section of the Scholastic Aptitude Test for school years 1985 through 1988, while in the math section the test scores varied--but they were still above the national average in 1988. According to the GAO, the average performance of DOD students on the American College Testing for the 1987-1988 school year exceeded the national average in all subject areas. The GAO concluded that, although the students scored well on the standardized achievement and aptitude tests, the scores are only Nowon pp 11~17 one measure of education quality. (pp. 15-19/GAO Draft Report) I DOD Response: Concur. FINDING C: The DOD Should Develop More Measures of Educational Quality. The GAO reported that the DOD annual report to the Congress on the quality of the education provided to its students is based primarily on student scores on the standardized achievement and aptitude tests (See Finding B). The GAO acknowledged that DOD students have scored well on these tests, but again emphasized that test scores alone do not provide a comprehensive measure of educational quality because, for example, military personnel are frequently reassigned during their careers. The GAO observed, therefore, that the test scores of students who attended the DOD schools can be expected to reflect the education received in other school systems, as well. The GAO concluded that, because of the influence of a variety of factors, the scores should be supplemented with indicators such as attendance rates, information on the courses taken by the students, promotion and drop-out rates, proportions of students meeting college and university entrance requirements, and student participation in the arts and extracurricular activities. The GAO pointed out that the DOD collects much of the data needed to develop additional indicators of educational quality--including attendance, academic progress, and grades. The GAO observed, however, that such available information is not routinely Now on pp 3-4. 16-17, and included in the annual report to the Congress. (pp. 3-4, 19 PP. 20-21, pp. 24-25/GAO Draft Report) DOD Response: Concur. The Dc3DS uses several methods and measures in its internal assessment of school effectiveness. Page 35 GAOjHRD-9tX13 DOD Overseas Schools Appendix III Comments From the Department of Defense 4 FINDING E: The DOD Needs Better Procedures To Document That ?;eachers Are Qualified. The GAO reported that a review of 225 teacher files showed that official collese transcriuts and complete Federal employment applications-were frequently missing--even though Federal personnel regulations require that these documents be maintained. For example, the GAO sampled teacher files in Korea; Frankfurt, Germany: and the Philippines. The GAO reported finding that all of the teacher files sampled in Korea; 58 percent of the teacher files sampled in Frankfurt, Germany; and 83 percent of the teacher files sampled in the Philippines were missing official transcripts. The GAO reported that the reason most cited by school and personnel office officials for the missing documentation was delays in receiving employment documentation for newly hired or recently transferred teachers. The GAO also observed that the DOD requires teachers to be certified to teach their grade levels and subjects. The GAO did find that most of the teacher files it reviewed contained Now on p 4 and pp 18-20 evidence that teachers were certified. (p. 5, pp. 22-25/GAO Draft Report) DOD Response: concur. FINDING F: Advisory Committee Functions. The GAO reported that the Defense Dependents' Education Act of 1978 requires the DOD to establish school advisory committees to provide advice to school principals and installation commanders on school affairs. The GAO observed that these committees are composed of an equal number of parents, whose students are enrolled in the DOD schools, and full-time professional employees. The GAO further observed that these committees are authorized to provide advice and make recommendations on almost any school-related issue. Now on pp 4-5, 21, and 23 (p.5, pp. 26-27, p. 29/GAO Draft Report) DOD Response: Concur. FINDING G: Committee Influence On Curricula And Budgets Limited. The GAO reoorted that advisorv committees had been established for all of-the schools it visited. The GAO found, however, that the committees had tended to focus on school operational issues, such as lunch programs and pedestrian safety rather than reviewing and making recommendations on school curricula and budgets. The GAO noted that those issues which advisory committees have considered were not documented--so a determination could not be made on how many of the committees wese given the opportunity to comment on these matters. The GAO reported that available documentation showed that, during school year 1987-1988, only two of the advisory committees for schools in the Frankfurt, Philippines, and Korean districts made recommendations or advised school principals of budget matters. The GAO attributed the limited involvement by advisory committee Page 37 GAO/HRD-90.13 DOD Overs.x.s Schools Appendix III tbmmmts From the Depart mrnt of Defensr 6 FINDING El: Composition of School Advisory Committees. The GAO reported that parent and teacher members from 8 of the 18 school advisory commlttees it interviewed indicated that they had little influence at advisory committee meetings because of the presence of noncommittee participants--such as school principals and teacher union representatives. According to the GAO, this resulted in the perception that influence was exerted over the Nowon pp 5 and23 teacher members. (P. 5. PP. 28-29/GAO Draft Report) DOD Response: Partially Concur. The DOD concurs that some school advisory committee members perceive that they have limited influence on school programs. School advisory committee meetings are open to the public. Since open meetings are the normal condition of school advisory committee meetings in the United States, the DOD does not concur in the GAO finding that the influence of committee members is limited by the forum in which the meetings are held. The composition of school advisory committees is established by statute (i.e., Section 1410 of Public Law 95-561 (19781, 20 U.S.C. 5 928) and includes equal representation of parents and school employees plus the teacher union representative as a non-voting member. The DOD does not concur that the influence of school advisory committee members is limited by the presence of teacher union representatives. FINDING I: Drug And Alcohol Abuse Prevention Programs Have Been Implemented. The GAO reported that, because of concerns about drug and alcohol use in its schools, the DOD has implemented drug and alcohol abuse prevention programs for elementary and secondary students in all of its schools. The GAO also noted that the DOD is implementing a pilot drug abuse prevention Now on pp 5 and 25~26 program for fifth and sixth grade students. (p. 6, pp. 31-33/ GAO Draft Report) DOD Response: Concur. The DoDDS has implemented drug abuse prevention programs in all of its overseas schools. In a recent survey of parents, they reported very high levels of confidence in DoDDS ability to deal with drug and alcohol abuse problems and very low levels of concern about drug abuse in the schools (i.e., about 7 percent of parents reported concern with drug abuse in DoDDS as compared to approximately 30 percent of parents in the U.S. who reported concern with drug abuse in their schools on "The Annual Gallop Poll of the Public's Attitudes Toward the Public Schools" which was reported in September of 1988). FINDING J: Most Facilities Problems Corrected. The GAO reported that school facilities problems--such as inadequate space and emergency lighting, leaking roofs, and unattractive landscaping. have been cited in accreditation survey reports by DOD'S independent school accrediting association. The GAO review of the most recent accrediting reports and its inspection of the Page 39 GAO/HRDYO-13 DOD Overseas Schools Appendix ID Comments From the Departmrnt of Defense 8 DOD Response: concur. The DOD concurs in the need for greater assurance that (1) graduates meet minimum graduation requirements and (2) teachers meet DOD requirements to teach their subjects and grade levels. - Graduation Requirements. The DOD concurs that high school principals should document their rationale for granting waivers in students files. This is already required by Dependents Schools Regulation 2000.1, paragraph F.l.d, in order to ensure that such waivers are in the best interests of students. In February 1990, the Director, DoDDS, will send a memorandum to all DoDDS high school principals reminding them that they are required to document waivers in students' files. - Teacher Files. Teacher files are reviewed periodically by DOD personnel during their employment. Teacher records are screened prior to hiring and prior to reassignment to positions within the DoDDS in order to ensure that teachers meet the requirements to teach specific subjects and/or grade levels to which they are assigned. While employed with the DoDDS, the teacher official personnel files are maintained by civilian personnel offices of the various Military Departments. Apparently, these official personnel files were not as complete as they could be. The GAO drd report, however, that the file management deficiency had not resulted in any teachers being improperly assigned to sublects and/or grade levels for which they were unqualified. By April 15, 1990, the Assistant Secretary of Defense for Force Management and Personnel will issue a memorandum to all Military Departments requesting that they initiate appropriate action to ensure that official personnel files of teachers include Standard Form 171s and college/university transcripts as evidence that teachers meet DOD requirements to teach the subjects and/or grade levels to which they are assigned. RECOMMENDATION3: The GAO recommended that the Secretary of Defense require the Office of Dependents Schools to ensure that advisory committees are provided the opportunity to review and advise school principals on school policy issues--specifically including curricula and budgets--by requiring the committees to document that they have been given that opportunity and are aware that they can elevate unresolved concerns to school system Now on pp 23-24 management above the principal level. (p. 30/GAO Draft Report) DOD Response: Concur. The Director, DoDDS, will revise school advisory committee guidelines to have school advisory committee chairpersons certify that the committee has been informed that they have the opportunity to advise on policy issues, including curriculum and budgets, and that they can elevate unresolved concerns to school system management above the principal level. School advisory committee guidelines will be revised to include this provislon for certification in the 1990-1991 school year. - Pagr 41 GAO/HRD90-13 DOD Overseas Schools Appendix IV Major Contributors to This Report Joseph J. Eglin, Jr., Assistant Director, (202).245-9623 Human Resources William A. Schmidt. Assignment Manager Division, Wayne M. Dow, Operations Research Analyst Washington D.C. Elliott C. Smith, Evaluator-in-Charge Far East Office Darryl L. Lee, Site Senior Elizabeth M. Guran, Evaluator Linda K. Willard, Evaluator Kurt W. Kcrshow, Site Senior European Office Kirk R. Boyer, Eval~lato~ Page 42 GAO/HKDYO-I3 DOD CNerseas Schools Appendix III C’ommmts From the Depart mrnt of Defense 7 facilities at 30 schools with problems identified by the accrediting association indicated that the association did not consider most of the problems serious enough to detract from the quality of education. The GAO further found that the DOD had Now on pp 5 and 26-27 corrected over 70 percent of the reported problems. (P. 6, PP. 33-35/GAO Draft Report1 DOD Response: concur. The DoDDS has attempted to correct, as soon as possible, any deficiencies that might detract from the quality of education. Unfortunately, there are still some instances where the DoDDS has not been able to provide totally adequate facilities, due to inadequate resources, unforeseen changes in student enrollments, and time constraints involved in completing major/minor construction projects overseas. RECOMMENDATIONS RECOMMENDATION1: The GAO recommended that, in addition to standardized test scores, the Secretary of Defense require the Office of Dependents Schools to use other measures of education quality to assess its schools--such as (1) attendance, promotion, and drop-out rates; (2) the ranyes of courses students take; and (3) how successful students are in meeting college entrance Now on p 20 requirements. (p. 25lGAQ Draft Report) DOD Response: concur. In addition to standardized test scores, the Office of Dependents Schools will use additional measures of educational quality to assess its schools. In its next annual assessment report, the Offlce of Dependents Schools will provide additional information to the Congress on measures of educational quality such as (1) attendance rates and (2) enrollments in and types of courses offered. Specific additional measures are currently being identified. On January 25, 1989, the Deputy Secretary of Defense requested that the DoDDS be included in the U.S. Department of Education's State Education Performance Chart. Representatives of the Office of Dependents Schools provided additional information on this request to representatives of the Office of Planning and Evaluation Services and the National Center for Education Statistics and are presently waiting for a final decision as to whether the DoDDS will be included in the Chart. RECOMMENDATION2: The GAO recommended that the Secretary of Defense require the Office of Dependents Schools to ensure the implementation of procedures that require (1) school principals to document in a student file the rationale for each exception granted to minimum graduation requirements and (2) teacher files to include all documents needed to demonstrate they meet DOD Nowonp 20 requirements to teach their subjects and grade levels. (p. 25/ GAO Draft Report) page40 GAO/HRD-9013 DOD 0wrsea.s Schools Appendix III Comments From the Department of Defense 5 members in critiquing curriculum and budget matters to a tendency by principals to discourage committee discussions of these matters--and a lack of awareness by members that they had authority to address such issues. The GAO further reported that representatives of 8 of the 18 school advisory committees it interviewed were unaware of the formal process for elevating concerns that could not be resolved with school principals. The GAO did acknowledge that the DOD has initiated several actions to Now on pp 4-5 and 21-24 improve communications within the school community. (P. 5, PP. 26-27, p. 29/GAO Draft Report) DOD Response: Partially Concur. The DOD concurs that parents often perceive that they lack influence in the schooling provided their children overseas. The DOD has initiated efforts to increase communications between school personnel and members of the communities which they serve. In the previously-cited school report card, parents indicated that communications between schools and parents had improved during the period of the GAO review. The DOD has established school advisory committees at each school and installation where the DOD operates schools overseas. The composition of these committees is outlined in statute (i.e., Section 1410 of Public Law 95-561 (1978), 20 U.S.C. cj 928) and their functions clearly include providing advice to school principals on all aspects of school operations, including school curriculum and budgets. Training for school advisory committee members has been conducted and school prin- cipals are required to consult with their advisory committees on all aspects of school operations. School advisory committee audio visual training materials were prepared by the DoDDS and are being used in ongoing school advisory committee training efforts. These materials identify school curriculum and budget as appropriate issues for discussion at committee meetings and include an overview of the DOD procedures to elevate unresolved issues. The effectiveness of these training tapes and related materials was evaluated by surveying all school advisory committees at the end of school year 1988-89. The Defense Manpower Data Center analyzed the school advisory committee responses to the survey and reported that these materials were effective. Where school advisory committees perceive that they are unable to influence school policy, these DOD procedures provide for review by DoDDS management and advisory committees at successively higher levels within the DOD. At the end of each school year, school advisory committees are expected to submit an annual report of their activities and DoDDS management officials review these reports to ensure that the advice of school advisory committees on all aspects of school operations has been carefully considered. In these reports, "curriculum" is consistently mentioned as an item the Committees have discussed even though the school curriculum was given very high ratings by parents in the DoDDS school report card. - 1 Page 38 GAO/HRD-90-13 DOD Overseas Schools 3 In addition to those methods and measures outlined in the Findings, (1) a "parent report card" has been initiated and (2) participation in the Department of Education's "Wall Chart" which compares educational performance data among the various States has been requested, and (3) the kind of school/student performance data suggested in this finding is being collected. The parent report card is a survey among parents that asks for their opinion of school effectiveness overall and in several specific areas. System-wide and local school survey findings were published in each school community and are being used extensively by school management in the development of local school improvement plans. In its annual assessment report, the DoDDS has primarily provided information on student achievement, because that information is traditionally reported by school districts in the United States. FINDING D: The DOD Needs Better Procedures To Document That Students Meet Graduation Requirements. The GAO reported that a review of the transcripts for all students, who graduated from the Korea district in 1988, as well as samples of those who graduated from the Frankfurt and Philippines districts, indicated that all but 1 of 96 students from the Frankfurt district met minimum graduation requirements; however, 25 percent of the 1988 DOD high school graduates in Korea and 10 percent of the graduates in the Philippines did not meet the DOD minimum graduation requirements. The GAO explained that a school principal can grant waivers or permit students to substitute elective courses for required courses, thus qualifying a student for graduation. The GAO found, however, that the student file often lacked documentation to support the rationale for the waiver actions. The GAO commented that, while the exceptions may have been justified, in the absence of documentation supporting the waivers or substitutions, parents and DOD management have no assurance that the exceptions were justified and in the students' NW on pp 17 18 and 19 best interests. (p. 4, pp. 21-22, pp. 24-25/GAO Draft Report) 20 DOD Response: Partially Concur. The DOD concurs with the GAO observation that the absence of documentation for waivers in student files limits management's ability to ensure that such waivers are in the best interest of students. The GAO reported that students graduated from the DOD schools in Korea and the Philippines, after having completed required and elective courses which were evaluated as meeting DODDS graduation requirements by the school principals. These principals' actions were authorized by the DoD policy guidance, which is intended to ensure that school programs meet the individual needs of students. For example, in Korea the GAO report indicates that a large number of graduates had remedial English as one of their required language arts courses. These language arts courses were appropriate to the individual needs of students and were appropriately used to satisfy the DOD graduation requirements. I’agr 36 GAO,‘HRD-90-13 DOD Overseas Schools Appendix III Comments From the Department of Defense GAO DRAFT REPORT - DATED DECEMBER8, 1989 (GAO CODE 104617) - OSD CASE 7807-A "DOD OVERSEAS SCHOOLS: BETTER ASSURANCES OF EDUCATIONAL QUALITY NEEDED" DEPARTMENT OF DEFENSE COMMENTS * l * * t FINDINGS FINDING A: Background: DOD Overseas School System. The GAO reported that the DOD overseas school system was established by the Defense Dependents Education Act of 1978 to provide high quality elementary and secondary education to dependent children of military and civilian personnel in overseas areas. According to the GAO, the Department funds and operates 271 schools--which enroll 151,000 students. The GAO observed that the Department budgeted $665 million in FY 1987, and $755 million in FY 1988, for the operation of the overseas school system. The GAO explained that the DOD school system employs more than 13,000 employees, of which 9,500 are teachers. The GAO noted that the system performs most of its own budget, supply, and teacher recruitment activities, although it relies on the Military Departments for some logistical, financial, and personnel support on a reimbursable basis. The GAO testified on October 5, 1988, before the Subcommittee on Military Personnel and Compensation, House Armed Services Committee, on the DOD school system. The Now on p 2 and pp 8-9 current report expands on that testimony. L/ (PP. l-2, PP. 12-13/GAO Draft Report) DOD Response: Concur. FINDING B: How the DOD Assesses Its Schools. The GAO reported that the DOD assesses the quality of education provided to its students in the following ways: School accreditation: The DOD contracts with the North Central Association of Schools and Colleges which conducts periodic accreditation reviews to ensure that the schools provide educational programs of high quality for all students. Curriculum review and development: The DOD uses a 7-year curriculum development cycle to keep current with the latest trends in education and ensure the appropriateness of its programs. L/ GAO/T-HRD-89-1, "GAO Testimony on Overseas Department of Defense Dependents Schools," Dated October 5, 1988, (OSD Case 7807) Page 34 GAO/HRD-90-13 DOD Overseas Schools Appendix III Comments From the Department of Defense ASSISTANT SECRETARY OF DEFENSE WASCIINCTON Dc 201014000 0 8 FEE 1993 MI. Franklin Frazier Director of Education and Employment Issues Human Resources Division U.S. General Accounting Office Washington, D.C. 20548 Dear Mr. Frazier: This is the Department of Defense (DOD) response to the General Accounting Office (GAO) Draft Report, "DOD OVERSEAS SCHOOLS: Better Assurances of Educational Quality Needed," dated December 8, 1989, (GAO Code 104617, OSD Case 7807-A). The DOD generally concurs with the findings and recommendations. The DoD recognizes the importance of the GAO findings and recommendations and appreciates the assistance that the GAO has provided to the Department of Defense Dependents Schools (DoDDS). In noting the need for additional measures of quality in education, the GAO has acknowledged that DoDDS students perform above the national averages on standardized achievement tests and on the Scholastic Aptitude Tests. The DOD would add that the most recent report of Scholastic Aptitude Test data shows that 60 percent of the DoDDS seniors took the tests, placing the DoDDS in the top 10 in the ranking of States in the Nation in percentage of students tested. In relation to those high participation States, the DoDDS ranked third in mathematics and second in verbal skills. The DOD concurs with the GAO recommendations that the DoDDS provide parents and school system management with additional indicators that their schools are providing students a high quality education. To meet this need, in January 1989, the Deputy Secretary of Defense requested that the Secretary of Education include the DoDDS on the State Education Performance Chart, which includes several measures of student performance. The DoDDS has provided the Department of Education staff with data and other information the Department requires to decide the feasibility of including the DoDDS in the Chart. In the mean- time, the DOD will pursue the objective of reporting additional indicators of educational quality. The most noteworthy effort to provide additional indicators of quality during the past year has been the "Parent Report Card," a survey mailed to every parent with a child in the DoDDS, J page32 GAO/HRD-BO-13DODOv~neasSchools Appendix II Scope and Method&@ in analyzing teacher files to ascertain, among other things, evidence of education, training, and certification. The districts included in our review, the teacher universes in each district at the time of our review, and the sample size of the teacher files reviewed are shown in table 11.2. Table 11.2:Teacher Universes Universe of Germany region teachers Sample size Frankfurt dlstmt 595 100 Pacific Region Korea district 60 60 Phlllppmes dlstrlct 279 65 T&al 934 225 Because our work in Korea was limited to the two small schools and two unit schools, we reviewed all of the teachers’ files. We randomly sam- pled the universe of teachers’ files at the Frankfurt and Philippines dis- tricts using a computer-generated list of random numbers. Our Frankfurt district sample consisted of 19 of the 23 schools in the district because the DOD Germany region inadvertently excluded 4 schools from the list of full-time teachers. The universe from which our sample was selected covers 83 percent of the schools and 85 percent of the full-time teachers. We used the results of our review of student and teacher records to esti- mate for each district the percentage of students not meeting graduation requirements and the percentage of incomplete teacher files. Because our estimates for Frankfurt and the Philippines districts are based on samples, each estimate has a sampling error associated with it. The sam- pling error for each estimate is at the 95-percent confidence level. To address DOD’S responsiveness to parental concerns, we interviewed 18 groups of parents and teachers who are members of the school advisory committees for the schools visited to obtain their views and concerns on the effectiveness of the committees. Although this was not a statistically representative sample, their remarks are examples of the views and con- cerns of parent and teacher school advisory committee members in these locations. We also met with military installation commanders or their representatives and local school officials to discuss their views on the committees’ effectiveness. We reviewed the minutes of school and instal- lation advisory committee meetings to determine the issues discussed and attendance at committee meetings. Page 30 GAO/HRD90-13 DOD Overseas Schools Appendix I Office of Dependents Schools:Schoolsand l3nrollments by Region Number Number Enrollment as Responsibility by of of of September Region/location country districts schools 1980 Atlanttc Unrted Krngdom, 4 42 16,187 (Eastcote, England) Norway, Bermuda, Iceland, Cuba, Belgrum, Netherlands, Canada (tncludtng Newfoundland), and West Indies Germany West Germany 8 141 87,861 (Wtesbaden, Germany) Medrterranean Spatn, Greece, Turkey, 3 35 13,296 (Madnd, Spatn) Bahratn, Italy, and Portugal (rncludtng Azores) Pacrfrc Japan, Oktnawa, Korea, 4 41 27,702 (Oktnawa. Japan) and the Philippines __~. ..--~~ Panama Panama 0 12 6,058 (Albrook Atr Force Statton, Panama) Total 19 271 151,184 Page 28 GAO/HRD9013 DOD Overseas Schools Chapter 4 DOD Has Been Responsive to Substance Abuse and Facilities Problems The Germany region also implemented the “Together” program, includ- ing “Choosing for Yourself,” by arranging training for teams of teachers, counselors, nurses, and principals. For example, during school year 1985-86, the developers of the “Together” program trained teams from all of the region’s junior and senior high schools. While the Germany region was not selected for initial implementation of “Here’s Looking at You, 2000,” it is piloting “Drug Abuse Resistance Education” (DARE) in the fifth and sixth grades. DARE is widely used in I J.S. school systems and was developed by the Los Angeles Police Department and the Los Angeles Unified School District. The program uses law enforcement personnel to instruct students on how to resist peer pressure to use drugs, exploring ways to say “no” when confronted or encouraged to use drugs, and practicing appropriate decision-making skills. Thirteen Germany region schools completed the pilot program in school year 1987-88. As a result of the pilot’s success, DOI)extended DAMS to 42 additional Germany region schools and 14 Atlantic region schools in 1988-89. DOD plans to implement D.~KEin all its schools by school year 1990-1991. if funds are available. Our review of NCA’Sevaluations of school facilities and our observation Most School Facilities of the facilities showed that I)ODgenerally corrects problems brought to Problems Were its attention. Corrected We visited 30 schools with facilities problems identified by the accredit- ing agency-22 in the Korea. Okinawa, and Philippines districts and 8 in the Frankfurt district. Of the 30 schools, 29 met the NM’s facilities stan- dards in spite of the noted problems. One failed because the problems, including inadequate cafeteria and physical education facilities, were considered by NCA to be serious enough to detract from the quality of education. IIOD is taking actions to correct these problems The types of problems identified at the 29 schools that met the stan- dards included unattractive landscaping, limited storage space, leaky roofs, and inadequate emergency lighting. During our school visits, we determined if DOD had taken actions to correct the facilities problems noted in the reports. DOD had corrected all the identified problems at 13 of the 30 schools and over one-half of those at the remaining 17 schools. The following tables show, by district, the number of schools with facilities problems and the member of their problrms corrected and uncorrected. (See t,able 4.1,) Page 26 GAO/HRD-SO-13 DOD Overseas Schools Chapter 3 Advisory Committees Can Be More Effective given that opportunity and are aware that they can elevate unresolved concerns to school system management above the principal level. DODagreed with our recommendation. It noted that the Office of Depen- Agency Comments and dents Schools plans to issue guidelines before the 1990-91 school year Our Evaluation requiring the committees to document that they have been informed that they have the opportunity to review and advise on school policy issues, and that they are aware of the formal complaint process. (See p. 41.) Page 24 GAO/HRD-90.13 DOD Overseas Schools Chapter 3 Advisory Committees Can Be Mom Effective school year 1987-88, only 2 of the 37 advisory committees for schools in the Frankfurt, Philippines, and Korea districts made recommendations or advised school principals on budget matters. In addition, committee members we spoke with had concerns that were not resolved with school principals, but they did not raise them in writing to higher levels. Among the reasons cited by advisory committee members for their lim- ited involvement in critiquing curricula and budget matters were (1) a tendency by principals to discourage committee discussions of such mat- ters and (2) a lack of awareness by members that they had the authority to address curricula and budgets. Representatives from 8 of the 18 school advisory committees we interviewed said they were unaware of the formal process whereby they can express in writing their concerns, which they have been unable to resolve with school principals, and ele- vate them to higher 1~~1s in the school system for consideration and resolution. DOD has initiated several actions intended to improve communications with the school community, including: . Requiring regional directors to conduct regular meetings with the lead- ership of parent, teacher, and student associations in their regions. l Disseminating audiovisual training materials that describe committee responsibilities and the process for elevating concerns to schools for advisory committee members. . Requiring district superintendents to meet annually with parent and military representatives from each school and command in their district. . Requiring the Office of Dependents Schools to establish community panels-consisting of parents, teachers, military command representa- tives, and administrators-to provide advice and participate in inter- viewing and selecting superintendents and principals. In addition, after our October 5, 1988, testimony, in which we stated that military commanders or their representatives often failed to attend required installation advisory committee meetings, the Secretary of Defense reemphasized the need for commanders or their representatives to attend and participate in these meetings. Page 22 GAO/HRD90-13 DOD Overseas Schools Chapter 2 Additional Assurances of Educational Quality Needed - files contain current and complete documentation that students meet graduation requirements and teachers are qualified. We recommend that the Secretary of Defense require the Office of Recommendations Dependents Schools to: l Use, in addition to standardized test scores, other measures of education quality to assess its schools, such as attendance, promotion, and drop- out rates; the ranges of courses students take; and how successful stu- dents are in meeting college entrance requirements. . Ensure that procedures are implemented requiring school principals to document in students’ files the rationale for each exception granted to minimum graduation requirements. We also recommend i hat the Secretary of Defense ensure that teachers’ files include all documents needed to demonstrate that they meet DOI) requirements to teach their subjects and grade levels. DOD agreed with our recommendations and noted that it plans to (1) Agency Comments and include such other quality measures as attendance rates and types of Our Evaluation courses offered in its next annual report to the Congress, (2) send a reminder to principals that they are required to document all waivers of graduation requirements, and (3) request the military departments to ensure that personnel files contain evidence that teachers meet r)or) teaching requiremtants (See pp. 40-41.) Page 20 GAO/HRD-9s19 DOD Overseas Schools Chapter 2 Additional Assurances of Educational Quality Needed allowed to substitute remedial English courses for the required language arts (English, reading, speech, and journalism) courses, and others were allowed to substitute foreign language courses, which are electives, for the required courses in such subjects as art, music, humanities, drama, and dance. In two cases, students were permitted to graduate after they were inadvertently granted full credit for partially completed required courses. According to school principals, in most cases they granted waivers from or substitutions for the graduation requirements because they believed graduation was in the best interest of these students. While these excep- tions may have been justified, the files contained no documentation of the rationale for the actions. Documentation for the exceptions would provide management and parents better assurance that students were granted proper waivers of minimum graduation requirements. Documentation Missing on Our review of a sample of 225 teachers’ files showed that the files fre- quently lacked documentation that teachers were fully qualified. As Some Teachers’ shown in table 2.1, official college transcripts and complete federal Qualifications employment applications were frequently missing. Table 2.1: Documentation of Teacher Employment Qualifications in Personnel Percentage of files missing Files Complete Files Official employment District reviewed transcript application Frankfurt 100 58 48 Korea 60 100 27 Ph~lbppmes 6i 83 40 Federal personnel regulations require DOD to maintain official records documenting employees’ qualifications and employment history and specify that the official personnel folders be maintained by the appro- priate civilian personnel office-usually a centralized office servicing a military installation. Among the required documents are an official col- lege transcript and a caomplete federal employment application with evi- dence that the application has been reviewed in determining that the applicant has the ncc~ssary experience. The reason most often cited by school and personnel office officials for the missing documentation was delays in receiving employment docu- mentation for newly llired or recently transferred teachers. Page 18 GAO/HRD-90-13 DOD Overseas Srhools Chapter 2 Additional Assurances of Educational Quality Needed Figure 2.4: DOD Students’ ACT Scores Are Above Average (School Year 1987-88) 25 Score 24 23 22 16 English Math Social Studies Natural Science Reading Subject u Nation Dal DODis required to annually assess and report to the Congress its per- DOD Should Report formance in providing a high-quality education to its students. DOD cur- More Measures of rently meets the requirement by reporting student test scores on CTBS. S.4T, and ACT. While I)OI) students have scored well on these tests, the test Educational Quality scores alone do not provide a comprehensive measure of educational quality. For example. military personnel are reassigned frequently dur- ing their careers, and the test scores of their children who attend DOD schools can be expcct,ed to reflect, in part, the education they received in other school systems. In a 1988 report on improving school system accountability, the Depart- ment of Education recommended that school systems supplement the results of student test scores with other indicators to better assess school effectiveness.’ These indicators would include information on (1) the courses students take; (2) attendance, promotion and dropout rates; (3) proportions of students meeting college and university entrance Page 16 GAO/HRD-90.13 WD Overseas Schools Chapter 2 Additional Assurances of Educational Quality Needed Figure 2.2: DOD Students in Grades 7-11 Score Well on Achievement Tests (School 60 Percentile Year 1987-88) 65 Grade 7 Grade 8 Grade g Grade 10 Grade 11 u Reading m LanguageArts Math SAT and ACT are used to help predict high-school students’ success in col- lege. SAT measures verbal and mathematical abilities, and is used by many colleges and universities to assess students for admission. The scores on each test are reported separately on a scale of 200-800 points. ~\CT tests students in five academic areas: English, mathematics, social studies, reading, and natural science. i\CT scores are reported on a scale from 1 to 36 points. nor) students scored higher than the national average on the verbal sec- tion of the SAT for school years 1985 through 1988. (See fig. 2.3.) In the math section of the test, eon students’ scores varied, from higher than the national average in the 1985 school year to slightly below the national average in school year 1987, and again above the national aver- age in 1988. Page 14 GAO/HRD90-13 DOD Overseas Schools ____--- ~- -.__- .~__- Chapter 2 Additional Assurances of Educational Quality Needed identify and consider teacher concerns. They evaluate the effectiveness of the curricula in the sixth and seventh years. .~ .-- .~ Student Performance on not) students participate in the Comprehensive Test of Basic Skills (CTIM), a nationally rec@nized standardized achievement test, and t hc Standardized Tests Scholastic Aptitude Test (SAT) and American College Testing (;\(‘.I‘) pro- gram aptitude tests. The ~1‘~s assesses student proficiency in five curricula areas: reading, mathematics, languag(t arts, social studies, and science. The results arc used to identify student strengths and weaknesses and improve instrllc’- tional programs. The scores are reported on a scale of 1 to 99 with t ht, national median--th(> point above which one-half and below wlric+ on+ half of all students taking the test scores-being the 50th percentilr. During school year 1X37-88, DODadministered the YIW to more than 117,000 students in grades 1 through 11, although IXJDstudent,s do not participate in t.he social studies and science exams until grade 7. As shown in figures 2. I and 2.2, r)oo students scored highrr than thr, 50th percentile in all curric,ldum areas at all grade levels. Paye 12 GAO,‘HRD-90-13 DOD Ovrrsras Schools Chapter 1 Introduction advises the principal on school policies and programs. At each military installation, the advisory committee raises school-level concerns regard- ing administrative and logistical matters to the installation commander. In response to the House Armed Services Committee report accompany- Objectives, Scope, and ing the 1989 National Defense Authorization Act that requested our Methodology study, and subsequent discussions with its office, our objectives were to determine l the adequacy of information used by WD to assess the quality of educa- tion provided by the DOD schools, including such indicators as school accreditation, curricula, achievement and aptitude test scores, gradua- tion requirements and rates, and teacher qualifications: l the responsiveness of the school system to parental concerns; . the status of drug and alcohol abuse prevention programs; and . the physical condition of school facilities. We performed work at the Office of Dependents’ Schools headquarters in Alexandria, Virginia, and at its Germany and Pacific regions. We focused our efforts on these two regions because (1) they include 67 per- cent of the schools and 76 percent of the students in the system, and (2) the concerns that led to the Committee’s interest were first raised by parents in the Pacific Region. We obtained information and interviewed officials at the Philippines and Korea districts in the Pacific region, the Frankfurt district in the Ger- many region, and at 33 selected schools in these three districts. In the Pacific Region, we also observed the physical condition of school facili- ties in the Okinawa district. We interviewed the military commanders and other supporting military personnel at installations served by the schools and parent and teacher members of school advisory committees. We also reviewed teacher and student records at the installations and schools we visited. Appendix II provides more detail on our scope and methodology. Our work was conducted between July 1988 and May 1989 in accord- ance with generally accepted government auditing standards. Page 10 GAO/HRD-90-13 DOD Overseas Schools Chapter 1 Introduction The Department of Defense (DOD) funds and operates 271 schools for the education of military and civilian dependents located overseas. DOD'S budgets for operating and maintaining these schools-which enroll about 151,000 students in kindergarten through grade 12-was $665 million in fiscal year 1987 and $755 million in fiscal year 1988. In letters to the House Armed Services Committee and in hearings held by the Committee at military installations in the Pacific in November 1987, parents expressed concerns about the quality of education pro- vided by the DODschools and the parents’ inability to have meaningful impact on the policies and operation of the schools. Subsequently, the Committee-in its report accompanying the fiscal year 1989 National Defense Authorization Act-directed GAOto study the strengths and weaknesses of the schools. In October 5, 1988, testimony before the Subcommittee on Military Per- sonnel and Compensation, House Armed Services Committee, we pro- vided preliminary information on, among other things, the DODsystem's quality of education, teacher evaluation systems, and responsiveness to parental concerns.’ We reported that, in general, teachers were being certified and evaluated as required, and that drug and alcohol abuse programs were being implemented. However, we also reported that some students were graduating from high school without meeting DOD-estab- lished minimum graduation requirements, teachers’ qualifications files were often incomplete, and the system for responding to parental con- cerns may not be fully effective. This report expands on the testimony and includes the results of our subsequent work in Germany. The school system was established by the Defense Dependents Educa- Background tion Act of 1978 to provide a high-quality elementary and secondary education to dependent children of military and civilian personnel in overseas areas. The system is administered by DOD'SOffice of Depen- dents Schools through five regional offices and 19 districts (see fig. 1.1). ‘GAO Testimony on Oversras Ik~partmcnt of Defense Dependents Schools (GAO/T-HKD-89-1, Ott 6, 1988) Page 8 GAO/HRD-W-13 DOD Overseas Schools Contents Executive Summary 2 Chapter 1 8 Introduction 13wkground 8 Objectives. Swpc. wcl Ivlc~thodology 10 Chapter 2 I1 Additional Assurances of Educational Qua1it.y Needed I)OD Needs lkttcr I’~~o~~durt~sto Document That St,udents 17 Meet Graduatiotl lit~quircments and Ttwhers Arc Qualifit~d Conclusions 19 Recommendations 20 Agency Comments ant1 Our Evaluation 20 - Chapter 3 21 Advisory Committees Advisory Committw Functions 21 Committees’ Influence on Curricula and Budget Issues 21 Can Be More Effective Limited Composition of School Advisory Committees 23 Conclusions 23 Recommendation 23 Agency Comments and Our Evaluation 24 Chapter 4 25 DOD Has Been Drug and Alcohol Abuse Prevention Programs Ifave Been 25 Implemented Responsive to Most School Facilities Problems Were Corrected 26 Substance Abuse and Facilities Problems Appendixes Appendix I: Office of Dependents Schools: Schools and 28 Enrollments by Rc,gion Appendix II: Scope and Methodology 29 Appendix III: Comments From the Department of Defense 32 Appendix IV: Mz#r (‘ontributors to This Report 42 Page 6 GAO/HRD-90-13 DOD Overseas Schools Executive Summary DOD Should Use DOD'S annual assessment of educational quality in its schools is based primarily on these test scores. But standardized test scores alone are an Additional Measures of unreliable measure of the education quality provided by particular School Effectiveness schools. Such scores can be influenced by a variety of educational, socie- tal, and other factors, and should be supplemented with indicators, such as attendance rates, range of courses students take, and drop-out rates. These additional measures would provide a better assessment of the quality of the DOD system whose students are very transient and, thus, may have test scores that are strongly influenced by their prior educa- tional experiences in other school systems. (See pp. 16-17.) - Some Principals Fail to Twenty-five percent of the 1988 DOD high-school graduates in Korea and Document Waivers of 10 percent of such graduates in the Philippines did not meet the DOD minimum graduation requirements. School principals can grant waivers Graduation Requirements or permit students to substitute elective for required courses, and thus qualify for graduation. However, students’ files often lacked evidence of the reasons for these actions. Without documentation of the reasons for the waivers or substitutions, parents and DOD management have no assurance that the exceptions were justified and in the students’ best interests. (See pp. 17- 18. ) Evidence Often Missing Similarly, teachers’ files often lacked the documents required by DOD to confirm that they are qualified to teach their grade levels and subjects. That Teachers Are For example, all of the teachers’ files in the Korea district, 58 percent of Qualified those in Frankfurt, and 83 percent of those in the Philippines were miss- ing official transcripts, and many files in the three districts were missing official federal employment applications needed to verify qualifications. (See pp. 18-19,) Advisory Committees Although UOL) has established required school advisory committees at Often Lacked Influence each school and installation GAO visited, the committees have seldom exercised their statmory authority to advise school principals on curric- ula and budget issues. Members of many of the advisory committees GAO interviewed said that their influence on school operations was limited because school principals limited discussion of such matters at committee meetings; members were unaware that t.hcre was a mechanism for elevating unresolved concerns to management above the local school level, such as to district or rcgi(~nal offices; and page4 GAO/HRDYO-I3DOD Overseas Schools Executive Summary In 1988, the Departmc,nt, of Defense (DOD) spent about $755 million to Purpose operate 271 oversc’as schools attended by over 150,000 students who arc’ dependents of milit,ary and DOD civilian personnel located overseas. Although the system generally has well-qualified teachers and good facilities, in recent ytlwrs parents have raised a variety of concerns about the quality ot’ clducation provided to their children, and parents’ pcrceivcd lack of influcncc over school policies and operations. In response, the House llrmed Services Committee directed GAO to study the strengths and weaknesses of these schools. GAO was to determine ( 1) the adequacy of information used by DOD to assess the quality of education provided by the DOD schools, (2) the responsiveness of the system to parental concerns, (3) the status of drug and alcohol abuse prevention efforts, and (4) the status of efforts to cor- rect physical deficiencies in school facilities. The DOD overseas school system was established to provide high-quality Background education to students from kindergarten through grade 12. To help ensure that it meets its educational responsibilities, DOD is required by law to: . Assess the quality of education it provides to its students each year. It does so principally by comparing the scores of its students with state- side students on standardized achievement and college aptitude tests. . Establish school advisory committees to provide a forum for communi- cations between the school system and the many military communities it serves. These committ,ees are comprised of equal numbers of parents and teachers, and are established to advise and make recommendations to school principals on school operations, particularly curriculum and budget matters. GAO studied DOD schools in the Pacific region, where parents’ concerns initially surfaced, and the Germany region, which is the system’s larg- est. Together, they contain two-thirds of the system’s 271 schools and serve three-fourths of its students. Within these regions, GAO reviewed 33 schools in three dist.ricts--the Philippines, Korea, and Frankfurt. While DOD schools art’ accredited and their students tend to score well on Results in Brief standardized tests, MN) school management and parents should have additional assurances that the schools are providing students with high- quality education. These scores provide only one measure of education page2 GAO/HRD-SO-13WDOverseasSchools
DOD Overseas Schools: Additional Assurances of Educational Quality Needed
Published by the Government Accountability Office on 1990-03-15.
Below is a raw (and likely hideous) rendition of the original report. (PDF)