Broadcasts to Cuba: TV Marti Surveys Are Flawed

Published by the Government Accountability Office on 1990-08-09.

Below is a raw (and likely hideous) rendition of the original report. (PDF)

    ,   Y
                    United   States   General   Accounting   Office
                                                                        ‘c .
                    Report to the Chairman, Subcommittee
GAO                 on Oversight and Investigations,
                    Committee on Energy and Commerce,
                    House of Representatives

August      1990

                      TV Marti Surveys Are
                   I( Flawed                                                       s

                     RE!STRICTED--     Not to be released outside the
                     General Accounting OffIce unless specifically
                     approved by the Office of Congressional



- ____ _---   _---- - - --_-   -------   -----
                   United States
GAO                General Accounting Office
                   Washington, D.C. 20648

                   National Security and
                   International Affairs Division


                   August 9, 1990

                   The Honorable John D. Dingell
                   Chairman, Subcommittee on
                     Oversight and Investigations
                   Committee on Energy and Commerce
                   House of Representatives

                   Dear Mr. Chairman:

                   In response to your request, we have reviewed the U.S. Information
                   Agency’s (CTSIA) efforts to assess the effectiveness of TV Marti reception
                   in Cuba from March 27 to May 12, 1990. USIA conducted two surveys in
                   making its assessment. The objectives of our review were to determine if
                   the methodology used in these two surveys and the reporting of the
                   results were consistent with sound data-gathering techniques and sup-
                   ported valid conclusions. The surveys were conducted to obtain infor-
                   mation from Cuban visitors, emigres, refugees, and other visitors
                   arriving in Miami, Florida, between March 28 and May 12, 1990. We also
                   reviewed information from the U.S. Interest Section in Havana about the
                   effectiveness of Cuban efforts to jam the TV Marti broadcasts.

                   In 1988, Public Law loo-459 authorized USLAto establish television
Background         broadcasting to Cuba. Congress appropriated $23.3 million to establish
                   TV Marti and to test its capability to broadcast into Cuba. The legisla-
                   tion required a test be made to demonstrate that broadcasting to Cuba
                   would be feasible and not interfere with U.S. domestic broadcasting. The
                   administration conducted a go-day test. On March 27, 1990, USIA began
                   test broadcasts from TV Marti facilities in Florida. TV Marti program-
                   ming originated from Washington, D.C., where the signal was beamed
                   via satellite to Cudjoe Key, Florida, about 110 miles from Havana. The
                   signal was relayed to an airborne transmitter tethered about 10,000 feet
                   above the key. The test broadcasts were primarily aimed at the Havana
                   area between the hours of 3:45 a.m. and 6:45 a.m. Eastern Standard

                   USIA’S survey results are flawed because they make projections on the
Results in Brief   reception of TV Marti based on incorrect application of generally recog-
                   nized methodological techniques. Moreover, the survey results differ
                   widely from information reported by the U.S. Interest Section in

                   Page 1                                     GAO/NSIhD90-252   TV Marti   Broadcasts

                   Havana. Based on one of its surveys, L’S~ reported that of 424 respon-
                   dents who tried to tune in, 112 respondents, or 26 percent, were able to
                   view TV Marti for 5 minutes or more. Projections from this survey indi-
                   cated that 273,000 households in Havana and three western provinces.
                   and between 1 million to 1.3 million Cubans could have viewed TV LMani
                   during the test period. An earlier USIA survey reported that as many as
                   7.3 million Cubans could have viewed TV Marti. These projections are
                   invalid because they were not based on a random sample and did not
                   demonstrate the similarity of respondents to Cuban households or to the
                   Cuban population.

                   Contrary to the survey results, the U.S. Interest Section in Havana
                   reported that less than one percent of persons interviewed in Cuba had
                   been able to view TV Marti and that its monitoring of the broadcasts
                   showed that TV Marti was effectively jammed by the Cuban

                   To assess the Cubans’ viewership of TV Marti, USIAdeveloped two sets
USIA Surveys Are   of questionnaires to use in interviewing visitors arriving in Miami,
Methodologically   Florida, from Cuba. Both questionnaires and the reported results had
Flawed             methodological flaws that made the results invalid.

                   For the first questionnaire, prepared by USIA’SRadio Marti research and
                   policy staff in Miami, Florida, interviewers took hearsay information
                   and made no apparent attempt to verify it, and they did not provide
                   complete descriptions of the demographic characteristics of inter-
                   viewees. Most important, the survey was not a representative sample of
                   TV Marti’s primary target area. Instead, USIA took a nonrandom sample
                   by interviewing 1,018 Cuban visitors, emigres, refugees, and other visi-
                   tors arriving at Miami International Airport from March 28 to April 18,

                   Radio Marti’s report on the results of this first survey stated that TV
                   Marti was “received in practically all of the Cuban territory with
                   acceptable quality” and that the potential population receiving the
                   signal was about 7.3 million. USIAresearch staff also noted several
                   problems with the survey methodology and concluded that “the study
                   does not yield reliable and valid results.”

                   The second questionnaire developed by Radio Marti’s Audience
                   Research Staff in Washington, D.C., also attempted to correlate the

                   Page 2                                    GAO/NSIAD-90-262   TV Marti   Broadcasts
      -- \

             Miami Airport respondents with the entire Cuban population and house-
             holds The conclusions drawn from the results are invalid because LSIA
             did not take a random sample of the Cuban population or households.
             Using data from this questionnaire, USIAand a USIA contractor prepared
             reports that contain inappropriate statistical generalizations about the
             viewership of TV Marti.

             A preliminary report of the survey results, dated May 2 1, 1990, pro-
             jected that between 1 million and 1.3 million Cubans were able to receive
              TV Marti during the first 6 weeks of broadcasts. Projections were based
             on the comments of 112 respondents, who indicated that they were able
              to view TV Marti for 5 minutes or more. The 112 respondents were
              26 percent of 424 respondents who claimed that they had tried to view
              TV Marti.

             Using the same survey data, a USIAcontractor prepared a second report,
             dated June 4, 1990, which projected that 273,000, or 28 percent, of the
             households in the primary target area of Havana and three western
             provinces would have been able to receive TV Marti. In the Havana area
             alone, the report projected that 33 percent of households were able to
             receive TV Marti.

             Based on the surveys of persons arriving at Miami International Airport,
             the USIA contractor projected potential viewership, even though its
             report stated that such projections are not statistically appropriate and
             are biased. To make valid projections, a random sample of the popula-
             tion or households in the target area would be required. However, the
             two USIAreports acknowledge that a random sample of Cuban popula-
             tion and households was not possible because Cuba is a closed society.
             Further, the validity of these projections can be questioned because the
             June report provides data showing that 24 of the 112 respondents, or
             21 percent, said they watched TV Marti on 4 days when there was no
             broadcast due to bad weather conditions or technical problems.

             In the absence of a random sample, the persuasiveness of survey results
             depends on how completely the sample represents the Cuban population
             and households in the target area. However, the May 1990 report does
             not compare the Miami Airport respondents to the Cuban population,
             and the June 1990 report does not adequately demonstrate that the
             respondents were representative of Havana area households. In fact, the
             June report shows that the respondents were not representative in sev-
             eral respects. For example, 61 percent of the survey respondents were

             Page 3                                     GAO/NSW90-252   TV Marti   Broadcasta


                    55 or older and about one-third were political refugees or emigres. How-
                    ever, the report analysis shows that only 19 percent of the Cuban popu-
                    lation is 55 or older, and the report indicates that refugees or emigres
                    would be more interested and aware of TV Marti than the general popu-
                    lation. These two factors indicate that the respondents were not

                    Further, the June report makes projections to households of Havana and
                    three western provinces without providing evidence that respondents’
                    households were typical of households in the target area. Therefore, the
                    report projections do not provide persuasive evidence that the respon-
                    dents are representative of the demographic and geographic characteris-
                    tics of Havana and surrounding area households.

                    Throughout the test period, U.S. officials stationed at the U.S. Interest
Conflicting         Section in Havana monitored and reported on TV Marti reception, Their
Information on TV   information on the extent of viewership and reception differs signifi-
Marti Reception     cantly from the reported results of survey respondents in Miami.
                    According to the U.S. Interest Section, extensive monitoring of TV Marti
                    in the Havana area, trips to various parts of the country, interviews
                    with various persons in Cuba, and results from questionnaires provided
                    to visitors to the consulate show that the Cuban government was effec-
                    tively jamming TV Marti. In addition, the Section reported that less than
                    one percent of Cubans have actually viewed TV Marti without disrup-
                    tion, especially in the Havana area. Details on the extent of TV Marti
                    reception in Cuba are classified.

                    This information conflicts significantly with USIA’SMay report, which
                    projected 26 percent population viewership, and with the June report,
                    which projected 28 percent of household viewership in Havana and
                    three western provinces. USIA officials could not reconcile these differ-
                    ences. However, they believed that TV Marti was reaching some portion
                    of the Cuban population.

                    We met with officials and obtained documents and records from USIA
Scopeand            headquarters, the Voice of America, TV Marti, and the State Department
Methodology         in the United States and from the US. Interest Section in Havana. We
                    obtained and reviewed pertinent reports and information related to TV
                    Marti broadcasts to Cuba.

                    Page 4                                    GAO/NSLUbBlb252   TV Mnrti   Broadcasts

We analyzed the methodology and reports used by USIA to measure TV
Marti reception in Cuba to determine whether they were conducted in
accordance with sound methodological and sampling techniques. Such
techniques are subjects of a large body of literature. Our references
include works by Leslie Kish, W. Edwards Deming, and William G.
Cochran. The references discuss the limitations of nonrandom sampling
in projecting or generalizing study resu1ts.l

As requested, we did not ask USJAto provide written comments on a
draft of this report. However, we discussed the results of our work with
USIA officials and considered their comments in preparing this report.
Our work was conducted between April and July 1990 in accordance
with generally accepted government auditing standards.

As arranged with your office, unless you publicly announce its contents
earlier, we plan no further distribution of this report until 30 days from
its issue date. At that time, we will send copies to USIA’SDirector, the
Secretary of State, and other interested congressional committees.
Copies will also be made available to others on request.

Please contact me at (202) 275-4128 if you or your staff have any ques-
tions concerning this report. Major contributors to this report were
Jess T. Ford, Assistant Director; Marilyn Mauch, Assistant Director;
John Gallant, Evaluator-in-Charge; and Arthur James, Statistician.

Sincerely yours,

Joseph E. Kelley
Director, Security and International
  Relations Issues

‘L. Kish, Surve Sam lin   (New York: Wiley, 1966), pp. l&29; W. Deming, Sam le Desi in Busing
Research( &k-%&y,
             ew ark:        1960), p. 28; W. Cochran, sampling ‘I’echniques (lie% York%iley, 1977

Page s                                                  GAO/‘NSIAD@O-262     TV Marti   Broadcaste
    . , .-

             Requests for copies of GAO reports    should be sent to:

             U.S. General Accounting     Office
             Post Office Box 6015
             Gaithersburg, Maryland     20877

             Telephone   202-275-6241

             The fast five copies of each report   are free. Additional   copies are
             $2.00 each.

             There is a 25% discount    on orders for 100 or more copies mailed to a
             single address.

             Orders must be prepaid by cash or by check or money order made
             out to the Superintendent of Documents.
United States                             First-Class Mail
General Accounting    Office            Postage & Fees Paid
Washington,   D.C. 20548

           for Private   Use $300
                                          Permit No. GlOO