, Y United States General Accounting Office ‘c . : Report to the Chairman, Subcommittee GAO on Oversight and Investigations, Committee on Energy and Commerce, House of Representatives August 1990 BROADCASTSTo CUBA : TV Marti Surveys Are I( Flawed s RE!STRICTED-- Not to be released outside the General Accounting OffIce unless specifically approved by the Office of Congressional Relations. - GAO/NSIAD90-252 .;. . i - ____ _--- _---- - - --_- ------- ----- United States GAO General Accounting Office Washington, D.C. 20648 National Security and International Affairs Division B-240007 August 9, 1990 The Honorable John D. Dingell Chairman, Subcommittee on Oversight and Investigations Committee on Energy and Commerce House of Representatives Dear Mr. Chairman: In response to your request, we have reviewed the U.S. Information Agency’s (CTSIA) efforts to assess the effectiveness of TV Marti reception in Cuba from March 27 to May 12, 1990. USIA conducted two surveys in making its assessment. The objectives of our review were to determine if the methodology used in these two surveys and the reporting of the results were consistent with sound data-gathering techniques and sup- ported valid conclusions. The surveys were conducted to obtain infor- mation from Cuban visitors, emigres, refugees, and other visitors arriving in Miami, Florida, between March 28 and May 12, 1990. We also reviewed information from the U.S. Interest Section in Havana about the effectiveness of Cuban efforts to jam the TV Marti broadcasts. In 1988, Public Law loo-459 authorized USLAto establish television Background broadcasting to Cuba. Congress appropriated $23.3 million to establish TV Marti and to test its capability to broadcast into Cuba. The legisla- tion required a test be made to demonstrate that broadcasting to Cuba would be feasible and not interfere with U.S. domestic broadcasting. The administration conducted a go-day test. On March 27, 1990, USIA began test broadcasts from TV Marti facilities in Florida. TV Marti program- ming originated from Washington, D.C., where the signal was beamed via satellite to Cudjoe Key, Florida, about 110 miles from Havana. The signal was relayed to an airborne transmitter tethered about 10,000 feet above the key. The test broadcasts were primarily aimed at the Havana area between the hours of 3:45 a.m. and 6:45 a.m. Eastern Standard Time. USIA’S survey results are flawed because they make projections on the Results in Brief reception of TV Marti based on incorrect application of generally recog- nized methodological techniques. Moreover, the survey results differ widely from information reported by the U.S. Interest Section in Page 1 GAO/NSIhD90-252 TV Marti Broadcasts - 5240007 Havana. Based on one of its surveys, L’S~ reported that of 424 respon- dents who tried to tune in, 112 respondents, or 26 percent, were able to view TV Marti for 5 minutes or more. Projections from this survey indi- cated that 273,000 households in Havana and three western provinces. and between 1 million to 1.3 million Cubans could have viewed TV LMani during the test period. An earlier USIA survey reported that as many as 7.3 million Cubans could have viewed TV Marti. These projections are invalid because they were not based on a random sample and did not demonstrate the similarity of respondents to Cuban households or to the Cuban population. Contrary to the survey results, the U.S. Interest Section in Havana reported that less than one percent of persons interviewed in Cuba had been able to view TV Marti and that its monitoring of the broadcasts showed that TV Marti was effectively jammed by the Cuban government. To assess the Cubans’ viewership of TV Marti, USIAdeveloped two sets USIA Surveys Are of questionnaires to use in interviewing visitors arriving in Miami, Methodologically Florida, from Cuba. Both questionnaires and the reported results had Flawed methodological flaws that made the results invalid. For the first questionnaire, prepared by USIA’SRadio Marti research and policy staff in Miami, Florida, interviewers took hearsay information and made no apparent attempt to verify it, and they did not provide complete descriptions of the demographic characteristics of inter- viewees. Most important, the survey was not a representative sample of TV Marti’s primary target area. Instead, USIA took a nonrandom sample by interviewing 1,018 Cuban visitors, emigres, refugees, and other visi- tors arriving at Miami International Airport from March 28 to April 18, 1990. Radio Marti’s report on the results of this first survey stated that TV Marti was “received in practically all of the Cuban territory with acceptable quality” and that the potential population receiving the signal was about 7.3 million. USIAresearch staff also noted several problems with the survey methodology and concluded that “the study does not yield reliable and valid results.” The second questionnaire developed by Radio Marti’s Audience Research Staff in Washington, D.C., also attempted to correlate the Page 2 GAO/NSIAD-90-262 TV Marti Broadcasts -- \ rK24ooo7 Miami Airport respondents with the entire Cuban population and house- holds The conclusions drawn from the results are invalid because LSIA did not take a random sample of the Cuban population or households. Using data from this questionnaire, USIAand a USIA contractor prepared reports that contain inappropriate statistical generalizations about the viewership of TV Marti. A preliminary report of the survey results, dated May 2 1, 1990, pro- jected that between 1 million and 1.3 million Cubans were able to receive TV Marti during the first 6 weeks of broadcasts. Projections were based on the comments of 112 respondents, who indicated that they were able to view TV Marti for 5 minutes or more. The 112 respondents were 26 percent of 424 respondents who claimed that they had tried to view TV Marti. Using the same survey data, a USIAcontractor prepared a second report, dated June 4, 1990, which projected that 273,000, or 28 percent, of the households in the primary target area of Havana and three western provinces would have been able to receive TV Marti. In the Havana area alone, the report projected that 33 percent of households were able to receive TV Marti. Based on the surveys of persons arriving at Miami International Airport, the USIA contractor projected potential viewership, even though its report stated that such projections are not statistically appropriate and are biased. To make valid projections, a random sample of the popula- tion or households in the target area would be required. However, the two USIAreports acknowledge that a random sample of Cuban popula- tion and households was not possible because Cuba is a closed society. Further, the validity of these projections can be questioned because the June report provides data showing that 24 of the 112 respondents, or 21 percent, said they watched TV Marti on 4 days when there was no broadcast due to bad weather conditions or technical problems. In the absence of a random sample, the persuasiveness of survey results depends on how completely the sample represents the Cuban population and households in the target area. However, the May 1990 report does not compare the Miami Airport respondents to the Cuban population, and the June 1990 report does not adequately demonstrate that the respondents were representative of Havana area households. In fact, the June report shows that the respondents were not representative in sev- eral respects. For example, 61 percent of the survey respondents were Page 3 GAO/NSW90-252 TV Marti Broadcasta _.. - E-240007 55 or older and about one-third were political refugees or emigres. How- ever, the report analysis shows that only 19 percent of the Cuban popu- lation is 55 or older, and the report indicates that refugees or emigres would be more interested and aware of TV Marti than the general popu- lation. These two factors indicate that the respondents were not representative. Further, the June report makes projections to households of Havana and three western provinces without providing evidence that respondents’ households were typical of households in the target area. Therefore, the report projections do not provide persuasive evidence that the respon- dents are representative of the demographic and geographic characteris- tics of Havana and surrounding area households. Throughout the test period, U.S. officials stationed at the U.S. Interest Conflicting Section in Havana monitored and reported on TV Marti reception, Their Information on TV information on the extent of viewership and reception differs signifi- Marti Reception cantly from the reported results of survey respondents in Miami. According to the U.S. Interest Section, extensive monitoring of TV Marti in the Havana area, trips to various parts of the country, interviews with various persons in Cuba, and results from questionnaires provided to visitors to the consulate show that the Cuban government was effec- tively jamming TV Marti. In addition, the Section reported that less than one percent of Cubans have actually viewed TV Marti without disrup- tion, especially in the Havana area. Details on the extent of TV Marti reception in Cuba are classified. This information conflicts significantly with USIA’SMay report, which projected 26 percent population viewership, and with the June report, which projected 28 percent of household viewership in Havana and three western provinces. USIA officials could not reconcile these differ- ences. However, they believed that TV Marti was reaching some portion of the Cuban population. We met with officials and obtained documents and records from USIA Scopeand headquarters, the Voice of America, TV Marti, and the State Department Methodology in the United States and from the US. Interest Section in Havana. We obtained and reviewed pertinent reports and information related to TV Marti broadcasts to Cuba. Page 4 GAO/NSLUbBlb252 TV Mnrti Broadcasts B24ooo7 We analyzed the methodology and reports used by USIA to measure TV Marti reception in Cuba to determine whether they were conducted in accordance with sound methodological and sampling techniques. Such techniques are subjects of a large body of literature. Our references include works by Leslie Kish, W. Edwards Deming, and William G. Cochran. The references discuss the limitations of nonrandom sampling in projecting or generalizing study resu1ts.l As requested, we did not ask USJAto provide written comments on a draft of this report. However, we discussed the results of our work with USIA officials and considered their comments in preparing this report. Our work was conducted between April and July 1990 in accordance with generally accepted government auditing standards. As arranged with your office, unless you publicly announce its contents earlier, we plan no further distribution of this report until 30 days from its issue date. At that time, we will send copies to USIA’SDirector, the Secretary of State, and other interested congressional committees. Copies will also be made available to others on request. Please contact me at (202) 275-4128 if you or your staff have any ques- tions concerning this report. Major contributors to this report were Jess T. Ford, Assistant Director; Marilyn Mauch, Assistant Director; John Gallant, Evaluator-in-Charge; and Arthur James, Statistician. Sincerely yours, Joseph E. Kelley Director, Security and International Relations Issues ‘L. Kish, Surve Sam lin (New York: Wiley, 1966), pp. l&29; W. Deming, Sam le Desi in Busing Research( &k-%&y, ew ark: 1960), p. 28; W. Cochran, sampling ‘I’echniques (lie% York%iley, 1977 Page s GAO/‘NSIAD@O-262 TV Marti Broadcaste . , .- 1 Requests for copies of GAO reports should be sent to: U.S. General Accounting Office Post Office Box 6015 Gaithersburg, Maryland 20877 Telephone 202-275-6241 The fast five copies of each report are free. Additional copies are $2.00 each. There is a 25% discount on orders for 100 or more copies mailed to a single address. Orders must be prepaid by cash or by check or money order made out to the Superintendent of Documents. --.I,*. United States First-Class Mail General Accounting Office Postage & Fees Paid Washington, D.C. 20548 Official Penalty Business for Private Use $300 I GAO Permit No. GlOO I
Broadcasts to Cuba: TV Marti Surveys Are Flawed
Published by the Government Accountability Office on 1990-08-09.
Below is a raw (and likely hideous) rendition of the original report. (PDF)