oversight

Physician Performance: Report Cards Under Development but Challenges Remain

Published by the Government Accountability Office on 1999-09-30.

Below is a raw (and likely hideous) rendition of the original report. (PDF)

                  United States General Accounting Office

GAO               Report to Congressional Requesters




September 1999
                  PHYSICIAN
                  PERFORMANCE
                  Report Cards Under
                  Development but
                  Challenges Remain




GAO/HEHS-99-178
      United States
GAO   General Accounting Office
      Washington, D.C. 20548

      Health, Education, and
      Human Services Division

      B-281938

      September 30, 1999

      The Honorable James M. Jeffords
      Chairman
      Committee on Health, Education, Labor, and Pensions
      United States Senate

      The Honorable Joseph I. Lieberman
      United States Senate

      While information has become available comparing health plan benefits,
      costs, customer satisfaction, and quality, it is not clear that this
      information is meeting consumers’ needs or expectations. With plans
      having overlapping panels of physicians and hospitals, it has become
      increasingly difficult for consumers to differentiate plans. Moreover, few
      consumers receive information on which to base one of their biggest
      health care decisions—their choice of doctors. Recognizing this, some
      organizations are attempting to measure and report on the performance of
      physicians and physician groups—with the hope that the results can be
      used to compare the quality of their care and services. How well physician
      and physician group performance measures assist consumers to make
      choices and how well they drive improvements in the health care market
      is unknown. Because of the growing interest in promoting informed health
      care decisions through public dissemination of performance information,
      you asked us to examine (1) the issues involved in measuring and
      reporting on physician and physician group performance, (2) current
      efforts to develop physician report cards, and (3) initiatives under way
      that may address impediments to measuring physician and physician
      group performance.

      To meet your request, we interviewed officials of purchasing groups,
      health plans, accreditation agencies, and federal programs; experts in
      health care performance measurement; and representatives from
      organizations that have formed to advance performance measurement. We
      visited three large health care purchasers—the Pacific Business Group on
      Health (PBGH), the Buyers Health Care Action Group (BHCAG), and the
      Health Care Financing Administration (HCFA)—and we interviewed two
      private health plans that publish physician group report cards—PacifiCare
      and Health Net—to discuss their efforts to measure and report on
      physician performance. We also reviewed report cards on cardiac
      surgeons issued by New York and Pennsylvania state agencies. We based
      our selection of purchaser groups and state initiatives on their reputations




      Page 1                                   GAO/HEHS-99-178 Physician Report Cards
                   B-281938




                   as innovators in the area of consumer health care information. The health
                   plans we selected have publicly reported the results of their comparisons
                   of physicians in their networks. We performed our work from
                   January 1999 through August 1999 in accordance with generally accepted
                   government auditing standards.


                   Measuring performance in health care is challenging in terms of
Results in Brief   identifying measures that truly reflect the quality of care individuals
                   receive. It is also difficult to make comparisons across plans or providers
                   that account for differences in the patients whom they treat that can affect
                   health care outcomes. Measuring the performance of physician groups and
                   individual physicians is even more difficult. Individual physicians or
                   groups perform a wide variety of services and typically perform any
                   individual service for a small number of patients. Only a fraction of these
                   services can be clearly linked to a measurable outcome. To make
                   meaningful comparisons among physicians, analysts must adjust any
                   measure selected to take into consideration the extent to which a
                   characteristic like the severity of a medical condition affects the outcomes
                   from care. To avoid these difficulties, current approaches to performance
                   measurement generally focus on physician groups instead of individual
                   physicians, and they measure processes such as whether services are
                   provided in accordance with agreed-upon norms rather than outcomes of
                   care. Adding to the challenges, however, are concerns that consumers
                   have regarding the privacy of their personal medical information and that
                   physicians have regarding the accuracy of performance measurement
                   data.

                   Even though the data and measures that are currently available are
                   limited, several different private and public organizations have developed
                   physician and physician group report cards using these data and measures.
                   For example, two purchasing groups and two California health plans are
                   avoiding some problems associated with measuring the performance of
                   individual physicians (such as small sample sizes) by reporting on the
                   performance of physician groups. In addition, in New York and
                   Pennsylvania, state agencies that have reported on the performance of
                   individual cardiac surgeons since the early 1990s have reported improved
                   performance scores since they began publishing them. While significant,
                   these efforts at physician report cards are in their early stages or are
                   limited in scope, and difficulties remain. For example, medical group
                   report cards provide information that is closer to the level of the individual
                   physician than health plan report cards do but, depending on the size of



                   Page 2                                    GAO/HEHS-99-178 Physician Report Cards
             B-281938




             the medical group, may not be very helpful for making an informed choice
             of a physician. In addition, questions about the accuracy and completeness
             of the data and the adequacy of the risk adjustment methodology limit
             consumer and physician confidence in the report cards.

             Some organizations are collaborating to develop more comprehensive,
             standardized performance measures and to facilitate the exchange of
             clinical and administrative data between physicians, plans, and
             purchasers. For example, several national accreditation organizations
             have formed a council to develop common performance measures. At the
             federal level, the Department of Health and Human Services (HHS) is
             working on a performance measurement system for its Medicare
             fee-for-service program and has been supporting research and working
             with other organizations to develop physician performance measures. In
             addition, HHS is establishing standards for administrative claims and
             encounter data as well as unique identifiers for individuals, plans, and
             providers—efforts that should help HHS and others in their performance
             measurement efforts.


             To date, most performance information has provided only data on health
Background   plans as a whole. Changes in the health care market—particularly the
             growth in the size of plans, the shifting of greater financial risk to
             physicians or physician groups, and the requirement in some cases that
             beneficiaries receive all their care from selected physicians within a
             plan—have made plan comparisons less useful for many consumers. Many
             consumers do not get to choose their health plan and even for consumers
             who can choose among plans, an individual physician’s performance may
             deviate greatly from the health plan’s average. These and other factors
             have prompted calls for physician report cards that can help consumers
             select physicians from those available within their health plan.

             Report cards are generally publicly released reports on the quality of care
             that provide comparative information on plan characteristics and
             performance. One widespread report card for health plans is prepared by
             the National Committee on Quality Assurance (NCQA). NCQA uses its Health
             Plan Employer Data and Information Set (HEDIS) to report on plan
             performance. HEDIS includes more than 60 performance indicators
             covering quality, access to and satisfaction with care, membership and use
             of services, finance, and management.




             Page 3                                   GAO/HEHS-99-178 Physician Report Cards
B-281938




When the development of health plan report cards began, plans were
expected to differ significantly in their provider networks, organizational
structure, and philosophical orientation, and the differences were
expected to be reflected in the overall quality of the plans. But the
marketplace has not evolved this way. Instead, to attract members and
gain market share, health plans began building larger, often overlapping
networks that offer consumers more provider choices than previously
available. With the same providers represented in two or more competing
plans, it has become increasingly difficult to differentiate between plans.
Thus, even for consumers who have a choice of plans, the comparative
plan information currently available may not demonstrate differences in
plan performance.

Physician practices are also undergoing significant change; more
physicians are joining medical groups, and these medical groups are
contracting with many health plans. The proportion of physicians in group
practices rose from approximately 11 percent in 1965 to 34 percent in
1995.1 In addition, according to the Medical Group Management
Association, its members contract with an average of 21 health
maintenance organizations (HMO) and preferred provider organizations
(PPO).2 As physician groups contract with more plans, individual plans may
have less influence over physician practice patterns and the quality of
services they provide, because any one plan may account for only a small
percentage of a medical group’s total volume of patients or income.

In addition, a growing number of physicians receive capitated payment—a
fixed monthly payment per patient—under which they accept financial risk
for providing a portion of or all patient care services. As plans shift more
financial risk to physician groups, a group’s economic incentive is to
minimize expensive services for sick patients. In a 1996-97 survey, more
than half of physicians (54 percent) reported that their practices received
capitation for some of their patients.3 In locations such as Seattle,
Washington, and Orange County, California, nearly three-fourths of
physicians reported receiving capitation for some of their patients. To the

1
 Henry J. Kaiser Family Foundation, Trends and Indicators in the Changing Health Care Market Place
(Menlo Park, Calif.: Aug. 1998).
2
 The Medical Group Management Association is a national professional and trade association. It
represents administrators of 7,491 medical group practices that included 181,974 physicians in 1997.
PPOs are similar to fee-for-service plans but provide enrollees a financial incentive—lower cost
sharing—to receive care from a network of providers that are normally reimbursed at a discounted
fee-for-service rate.
3
 Center for Studying Health System Change, Data Bulletin: Results from the Community Tracking
Study (Washington, D.C.: Fall 1997).



Page 4                                                  GAO/HEHS-99-178 Physician Report Cards
                        B-281938




                        extent that physician assumption of financial risk affects quality of care,
                        this trend further shifts the focus on quality from plans to physicians.

                        For most employed Americans, their employer determines the number and
                        type of health insurance plans available to them. For workers whose
                        employers do not offer a choice of plans, report cards that compare plans
                        have no utility. A 1997 survey conducted by the Research Triangle Institute
                        found that less than one in five—17 percent—of private employers that
                        offered insurance to their employees provided a choice among plans.4
                        Another study reported that of employers that offer health insurance, 92
                        percent of small firms and 44 percent of larger firms (those with more than
                        200 employees) offered only one plan in 1998.5 Counting employees rather
                        than employers, less than half—only 41 percent—of employees who are
                        offered health insurance can choose from two or more health plans.6
                        Health plan report cards may also be of little use to more than 14 million
                        of the country’s 40 million Medicare beneficiaries—those who did not have
                        a choice of managed care plans in 1998.7


                        The heterogeneity of health care makes performance measurement
Developing Physician    challenging in terms of identifying measures that truly reflect the quality of
Report Cards Is         care that individuals receive. Making valid comparisons across plans or
Challenging             providers that ultimately account for patient differences that affect
                        outcomes is also difficult. These challenges are magnified in attempts to
                        measure the performance of physician groups and individual physicians.
                        For example, unlike plans that have a large number of enrollees, individual
                        groups or physicians generally see a small number of patients with specific
                        conditions. These attempts are further complicated by a concern that
                        consumers and physicians have regarding the use of performance
                        measurement data in the first place.


Selecting Appropriate   Medicine involves a wide variety of services, only a portion of which can
Measures Is Difficult   be clearly linked to health outcomes. Health outcomes are also influenced

                        4
                         The Robert Wood Johnson Foundation, 1997 Employer Health Insurance Survey (Princeton, N.J.:
                        1997).
                        5
                         Henry J. Kaiser Family Foundation, Health Benefits of Small Employers in 1998 (Menlo Park, Calif.:
                        Feb. 1999), p. 18.
                        6
                         Robert Wood Johnson Foundation.
                        7
                         In 1998, 4 million Medicare beneficiaries had only one managed care plan available in their county and
                        10.6 million beneficiaries lived in counties with no plan at all. See Medicare Managed Care Plans: Many
                        Factors Contribute to Recent Withdrawals; Plan Interest Continues (GAO/HEHS-99-91, Apr. 27, 1999).



                        Page 5                                                  GAO/HEHS-99-178 Physician Report Cards
B-281938




by factors such as a patient’s age, medical history, and heredity. In
addition, most individual physicians perform any specific service a
relatively few times in a given year, making it more difficult to adjust for
differences in patients and creating other statistical problems. Efforts to
create report cards on health plans and providers are also complicated by
the different needs of consumers. Current attempts to resolve the
difficulties of physician performance measurement include aggregating
physicians into groups; focusing on certain types of physicians, such as
cardiac surgeons, who perform highly specialized services; and
substituting measures of process for measures of outcome.

To make meaningful comparisons among physicians, analysts must select
measures while taking into account factors that can affect the outcomes of
care, such as a patient’s medical history. For example, whether the patient
is treated for a first or second heart attack affects the likelihood of a
successful outcome from cardiac surgery. Measures that did not account
for differences in such factors would penalize physicians who treat the
sickest patients. Without proper adjustment in the measures, physicians
might choose to avoid high-risk patients in order to maintain higher
performance scores.8 Today, such adjustments are limited, and it may
never be possible to account for every characteristic of patients that could
affect their health outcomes.9

Another challenge in performance measurement is that different
consumers have different information needs that are not likely to be
adequately met all in a single physician report card. Consumers prefer
performance information that matches their own medical conditions and
situations. For the majority of consumers who are generally in good
health, clinical quality indicators may not be as relevant as service quality
indicators, such as the waiting times for an appointment. But for
individuals with chronic ill health, those who use physician services the
most, clinical quality measures may be more critical. It may not be
possible to measure and report on physicians at the level of detail that is
meaningful and useful to all consumers.


8
 The results of a recent study on physician profiles for patients with diabetes suggests that physicians
might refuse to care for sick patients, those who have failed therapy or those who do not adhere to
treatment plans, in order to improve their profile scores. See Timothy P. Hofer and others, “The
Unreliability of Individual Physician ‘Report Cards’ for Assessing the Cost and Quality of Care of a
Chronic Disease,” Journal of the American Medical Association, Vol. 281, No. 22 (1999), pp. 2098,
2104, and 2105.
9
 President’s Advisory Commission on Consumer Protection and Quality in the Health Care Industry,
Quality First: Better Health Care for All Americans, Final Report to the President of the United States
(Washington, D.C.: 1998).



Page 6                                                   GAO/HEHS-99-178 Physician Report Cards
                            B-281938




                            Current performance measurement practices make an effort to detect
                            physician and physician group practice differences, report on a variety of
                            indicators, and generally avoid the need to identify detailed patient
                            characteristics. One approach to solving the measurement problems
                            associated with individual physicians’ seeing only a small number of
                            patients with a given condition is to focus on physicians organized into
                            groups, so that the number of patients with a given medical condition is
                            high enough to provide meaningful data.

                            Most quality indicators in use today focus on process measures, such as
                            the percentage of women older than 50 in a plan who receive screening
                            mammograms. Many of the measures NCQA uses for health plans in HEDIS
                            are process measures. Measures of medical care process are popular in
                            part because the data required are obtained relatively easily from
                            administrative databases. Furthermore, process measures, particularly for
                            preventive services, avoid the difficulties inherent in trying to adjust the
                            results for differences in patient characteristics. But process measures
                            have many shortcomings: They capture a very limited range of medical
                            services, they tend to measure whether a service was provided when it
                            was called for but not how well it was done, and they focus heavily on
                            preventive care services because the universe of patients who should be
                            receiving them is most easily identified. Outcome measures, those that
                            indicate whether a patient’s health improved after care, generally remain
                            elusive.


Assembling Performance      Creating a physician performance measurement system involves collecting
Data Requires Cooperation   and verifying medical care data. In order to collect the data, the concerns
                            of consumers and physicians regarding how the data will be used must be
                            addressed. To ensure that measures of performance are accurate, the data
                            going into the measures must be verified and free from manipulation.

                            Physicians are concerned about the potential that inaccurate performance
                            scores will unfairly affect their practice. Physicians we interviewed told us
                            that issues of data quality and the appropriate attribution of performance
                            scores to individual physicians must be addressed before performance
                            measurement data are made public. Although administrative records, such
                            as claims for payment, are readily available for the fee-for-service sector,
                            they often do not include all the information that performance
                            measurement requires, such as a patient’s condition or the results of
                            services rendered, and for some indicators they are not collected because
                            they were created for billing purposes and not for performance



                            Page 7                                    GAO/HEHS-99-178 Physician Report Cards
B-281938




measurement. Medical records provide much more complete information,
but their analysis is expensive because few records are automated.
Gathering information through surveys falls somewhere between
administrative data and medical record review in terms of ease and
expense. One limitation of ratings of consumer satisfaction is that
consumers cannot always tell if the care they received was appropriate or
technically good; research has not shown a consistent relationship
between consumer satisfaction and the technical quality of care.

Another concern for physicians is the method of determining which
physicians should be held accountable for specific actions or outcomes.
Some physicians argue that it is difficult to fairly attribute a change in
health status to a particular physician because many other factors come
into play. For example, some patients may see a variety of physicians over
the course of a year, each potentially recommending or performing a
needed service. If a patient has not received a particular service, which
physician should be held accountable for the omission? Or a physician
may have recommended a very effective treatment to a patient, but that
patient’s condition did not improve because he or she did not comply with
the physician’s recommendation. The question of attribution becomes
even more difficult as systems of health care become more integrated and
a “team” of providers rather than one physician is responsible for
patients’ health care.

Because physicians control the majority of the data necessary to measure
their performance, these concerns must be addressed if measurement
efforts are to be successful. Physicians are responsible for coding
administrative data, whether they are for reimbursement for claims or
other data used by managed care plans. Physicians also maintain medical
records for individual patients. To automate these records in order to
make performance measurement data better would be expensive, and it is
unlikely that physicians or physician groups will provide the resources for
automation.

Ensuring that the data that are collected are accurate is another challenge.
In any measurement system, participants may manipulate the data to
improve their performance scores. For example, physicians could
exaggerate the severity of patients’ conditions to ensure a more favorable
rating. Or they could simply avoid taking on difficult cases in order to
improve their success rates. Preventing this sort of manipulation requires
activities such as auditing the data by comparing them to medical records.




Page 8                                   GAO/HEHS-99-178 Physician Report Cards
B-281938




Consumers are concerned about the privacy of their personal medical
information, and this concern may lead to rules that restrict efforts to
provide objective information on physician performance. One survey
found that no more than about one-third of adults in the United States
trust health plans and government programs to maintain confidentiality all
or most of the time.10 Meanwhile, consumers want an unbiased, expert
source of information about health care quality.11 State laws vary
significantly, but in some states efforts to protect the privacy of medical
records could affect efforts to ensure that reported measures are
comparable and that the data are not manipulated. For example, in
Minnesota, any release of a patient’s health records for research purposes
requires, among other things, that the provider attempt to acquire the
patient’s consent and determine that individually identifiable records are
necessary, that the researcher’s safeguards are adequate, and that the
researcher will not use the records for purposes other than those in the
original request without the patient’s consent. According to a BHCAG
official, Minnesota’s state privacy laws forced the group to abandon its
attempts to collect HEDIS data from care systems and have hampered
efforts to obtain survey data regarding quality of care for people with
chronic conditions. Finding the appropriate balance between allowing
access to medical records to ensure reliable, unbiased information on
health care quality and maintaining privacy concerns is subject to
considerable debate.




10
 California Healthcare Foundation, Medical Privacy and Confidentiality Survey (Oakland, Calif.:
Jan. 28, 1999).
11
 Consumers do not necessarily trust currently available performance information. Three of 10
Americans surveyed said that information on quality of care from groups of doctors or state medical
societies had little credibility; 4 of 10 had little faith in information from government agencies, and
nearly 5 of 10 said that information from newspapers, television, and other media was not very
believable. See Agency for Health Care Policy and Research and Henry J. Kaiser Family Foundation,
Americans as Health Care Consumers: The Role of Quality Information, Highlights of a National
Survey (Washington, D.C.: 1996).



Page 9                                                   GAO/HEHS-99-178 Physician Report Cards
                            B-281938




                            Several private and public organizations are involved in a variety of
Report Cards on             activities to measure and report on physician performance. A purchaser
Physicians and              and two health plans in California and a purchaser in Minnesota have
Physician Groups            worked on moving performance measurement down to the level of the
                            medical group or independent practice association (IPA) or, in Minnesota, a
Indicate Progress but       care system.12 In New York and Pennsylvania, state agencies have
Their Usefulness            published risk-adjusted mortality rates for specific procedures performed
                            by cardiac surgeons. While the current report cards demonstrate that
Remains Limited             some reporting is possible, shortcomings in these physician and group
                            report cards include the doubtful value to consumers of scores for large
                            medical groups, questions about the quality and the expense of collecting
                            the data on which reporting is based, and inconsistencies among the
                            report cards.


Purchaser and Health Plan   Several organizations have developed report cards for physician groups.
Report Cards Compare        PBGH, two California health plans—PacifiCare and Health Net—and BHCAG

Physician Groups            in Minnesota have moved a step closer to reporting physician performance
                            by publishing report cards on medical groups and IPAs.

Pacific Business Group on   PBGH is a business coalition of 33 public and private purchasers of health
Health                      care representing more than 3 million employees, retirees, and
                            dependents. As physician networks overlapped more and differentiation in
                            California health plans blurred, PBGH started partnering with medical
                            groups and IPAs on quality improvement initiatives. Together, they
                            developed a publicly reportable measurement tool called the Physician
                            Value Check Survey. In 1996, the survey covered 49 California medical
                            groups (and 9 from the Pacific Northwest) that ranged in size from




                            12
                               A medical group is two or more doctors who work together to provide medical services to patients.
                            Typically, doctors who work in a medical group—both primary care doctors and specialists—share a
                            single office or several offices if the group is very large. An independent practice association is a
                            network of individual physicians who practice medicine by themselves or in small groups (often
                            composed of one type of doctor, such as pediatricians) and who join together as an association to
                            provide a range of primary and specialty care services to patients. Care systems began in 1997, when
                            the employer members of BHCAG began contracting with health care providers directly rather than
                            with health plans. The providers organized themselves into care systems, with some care systems
                            resembling multispecialty medical groups and others looking more like independent practice
                            associations.



                            Page 10                                                 GAO/HEHS-99-178 Physician Report Cards
             B-281938




             approximately 15,000 patients to more than 1 million patients. Responses
             were obtained from 31,000 patients.13

             PBGH chose a survey to collect data because it did not have the resources
             to mount a full-scale medical record review, and the survey was a less
             costly means of evaluating physician group performance and obtaining
             information on the consumers’ perspectives. So that PBGH could generalize
             the results to all patients seen by a medical group, PBGH and its partners
             drew patient samples from each medical group’s entire patient population
             rather than just from PBGH members. PBGH used the survey results to
             publicly compare medical groups in several areas: a summary report card
             with measures such as overall satisfaction, a preventive care services
             report card, and specific report cards on care for high blood pressure and
             high cholesterol. For each report card, it classified the groups into three
             categories: above average, average, and below average. (See the appendix
             for details from the PBGH report card.)

PacifiCare   PacifiCare of California is an HMO that has since 1998 produced a medical
             group report card called the Quality Index. The publicly reported Quality
             Index uses measures selected from PacifiCare’s internal provider profiles,
             which contain data on more than 60 clinical and service performance
             measures for its medical groups and IPAs.14 PacifiCare selected 14 of these
             measures for inclusion in the Quality Index.15 It based its selection on the
             preferences of focus groups of consumers and the extent to which the
             physicians could take actions that affected the measured activities.

             The information in the Quality Index is compiled from the health plan’s
             administrative databases, customer service department records, and
             enrollee satisfaction surveys. Thus, the Quality Index reflects the health
             care experiences and opinions of only PacifiCare enrollees. The Quality
             Index includes measures of clinical performance, service performance,

             13
               According to PBGH’s Director of Research, these 31,000 respondents represent more than 8 million
             enrollees in managed care plans. In 1996, the Physician Value Check Survey was sent to 1,000 patients
             between the ages of 18 and 70 randomly sampled in each medical group. The overall survey response
             rate was about 55 percent. PBGH administered the Physician Value Check Survey again in 1998;
             however, the results from this survey were not available at the time of our review. PBGH officials said
             they plan to release the 1998 Physician Value Check Survey results on September 23, 1999, with scores
             on the changes between 1996 and 1998—that is, to see whether for the same group of patients, the
             physician groups’ performance improved, worsened, or stayed the same over time.
             14
               PacifiCare profiles medical groups and IPAs that have about 500 or more PacifiCare enrollees. To be
             included in the Quality Index report, groups must have at least 1,000 PacifiCare commercial enrollees
             and 500 Secure Horizons enrollees (its Medicare managed care program). Using this methodology,
             PacifiCare is able to report on physician organizations that provide care to the majority of its enrollees.
             15
               The March 1999 Quality Index included 28 measures—14 for PacifiCare commercial enrollees and 14
             for Secure Horizons enrollees.



             Page 11                                                   GAO/HEHS-99-178 Physician Report Cards
             B-281938




             enrollee satisfaction, and administrative data submission. In 1999, the
             Quality Index included process measures such as “eye examinations for
             people with diabetes” that were not included in the 1998 report. The
             reported Quality Index scores are percentile ranks for medical groups or
             IPAs compared with all other groups. Groups ranking in the top
             10 percentile of a measure are considered “best practice” groups for that
             measure. According to PacifiCare’s Medical Director, improvement has
             occurred in several areas, such as mammography screening rates and
             retinal examinations for diabetics. (See the appendix for details from the
             PacifiCare Quality Index.)

Health Net   Health Net is another managed care plan in California with more than
             2.2 million enrollees. Its Participating Physician Group Report Card
             includes information for 131 medical groups in California, all of which are
             under performance-based contracts. In 1999, a percentage of each medical
             group’s payment is contingent on the quality of care it provides to
             enrollees, as measured by both their satisfaction ratings and other process
             measures.16 Thus, Health Net provides (1) information to the plan’s
             enrollees to encourage them to “vote with their feet” by migrating to the
             top performing groups and (2) a direct financial incentive for the medical
             groups that is associated with their performance. Health Net’s report card
             is derived from a satisfaction survey of the plan’s enrollees.17 It includes
             numerical scores representing the percentage of respondents who
             reported that they were satisfied regarding each of 17 measures. Health
             Net divides the medical groups into three categories for comparison:
             excellent, very good, and good. (See the appendix for details from the
             Health Net Participating Physician Report Card.)

             Health Net is also developing report cards on care for certain chronic
             conditions. For these, Health Net uses administrative data to identify
             enrollees with a given condition and sends them a standardized survey
             that measures such things as the number of work days lost to illness or
             injury. It also measures compliance with national guidelines on
             management of the condition or disease. Health Net published a report
             card on asthma care in December 1998 and is currently working on report

             16
               Health Net has three reward components to its performance-based contracts: (1) pay for excellence,
             given to the top 25 groups; (2) pay for performance, given to groups that exceed fixed performance
             targets; and (3) pay for improvement, given to groups that improve, even if they are ranked relatively
             low.
             17
              Health Net sent its 1998 enrollee satisfaction survey to more than 500,000 enrollee households in
             California. According to the President for Health Benchmarks, Inc. (the organization responsible for
             producing Health Net’s physician group report cards), the overall survey response rate was about 30 to
             40 percent. Only physician groups with 75 or more plan enrollees responding to Health Net’s
             satisfaction survey were included in the report card.



             Page 12                                                 GAO/HEHS-99-178 Physician Report Cards
                                B-281938




                                cards for diabetes and congestive heart failure.18 (See the appendix for
                                details from the asthma report card.)

Buyers Health Care Action       In 1997, the employer members of BHCAG began a program of contracting
Group                           with health care providers directly rather than with health plans. The
                                providers organized themselves into care systems, with some care systems
                                resembling multispecialty medical groups and others looking more like
                                IPAs.19 BHCAG set out to adopt HEDIS health plan measures for each care
                                system in the program. However, the purchasing group decided to
                                abandon its effort to use the HEDIS measures because the number of
                                patients within each care system who met the criteria for a particular
                                measure was too small for valid, comparative analysis. BHCAG was unable
                                to identify more than 100 plan enrollees for any of the measures in more
                                than one or two care systems. According to a BHCAG official, BHCAG also
                                decided not to develop the HEDIS data base for the care systems’ patient
                                population for two reasons. First, Minnesota’s medical record
                                confidentiality law prevented BHCAG from auditing the data to ensure their
                                accuracy. Second, because patients who are not associated with BHCAG
                                member firms are not necessarily obligated to seek primary care from a
                                single care system, it would be difficult to establish the base for many
                                HEDIS measures. Instead, BHCAG developed and distributed a satisfaction
                                survey to members’ employees and reported the results to its enrollees
                                and the general public.

                                In 1996-98, BHCAG reported data on 12 measures from the survey, focusing
                                on such issues as access to services and interactions with physicians.
                                Beginning in 1999, BHCAG adopted a modified version of the Consumer
                                Assessment of Health Plans survey developed by the Agency for Health
                                Care Policy and Research (AHCPR). While BHCAG has approximately 150,000
                                individuals enrolled in care systems, they are unevenly distributed: About
                                75 percent are enrolled in three larger care systems. To increase its sample
                                sizes for care outside the Minneapolis St. Paul metropolitan area, BHCAG
                                conducted the survey with Minnesota state employees, which increased
                                the total potential survey population from about 150,000 to about 300,000.

State Report Cards on Cardiac   Since the early 1990s, the New York Department of Health and the
Surgeons                        Pennsylvania Health Care Cost Containment Council have published

                                18
                                  Health Net’s Asthma Care Report Card ratings were based on a 1996 survey administered to more
                                than 32,000 Health Net enrollees who suffered from asthma. Because only California physician groups
                                with 35 or more plan enrollees responding to the survey were included in the report, the Asthma Care
                                Report Card included information for 47 medical groups.
                                19
                                 A characteristic of these care systems that sets them apart from many health plans is that primary
                                care physicians can belong to only one care system.



                                Page 13                                                 GAO/HEHS-99-178 Physician Report Cards
B-281938




physician-specific mortality rates for patients undergoing coronary artery
bypass graft (CABG) surgery.20 Because patients’ characteristics such as age
and other health problems play a large role in the rates of complications
and deaths associated with CABG surgery, efforts have been made to adjust
the performance measures for differences in patients’ conditions. For
example, the New York risk-adjustment process incorporates
approximately 40 risk factors for each patient. The New York Department
of Health also seeks to verify the data through activities such as
cross-matching cardiac surgery with other Department databases and
reviewing medical records for a sample of cases.

The New York Department of Health reported that the state’s CABG surgery
mortality rate dropped by more than 30 percent following the publication
of the report card, from 3.52 percent in 1989 to 2.44 percent in 1996.
Similarly, the Pennsylvania Health Care Cost Containment Council
reported that inhospital mortality was 22-percent lower in 1995 than it was
in 1991 (3.8 percent compared with 4.9 percent). However, these results
and the effect of the report cards have not been without controversy.

Some researchers assert that performance reporting has played a
significant role in the decline in the CABG surgery death rate. They point to
evaluations and improvements in CABG surgery processes, changes in
referral patterns–such as concentrating the most difficult cases with
top-performing physicians–and a reduction in the number of surgeons
who perform these procedures only a few times each year.21 Critics of the
New York program contend that performance reporting is not responsible
for a decline in the mortality rate. They claim other factors such as
surgeons’ electing not to operate on critically ill patients and possibly
referring high-risk cases to out-of-state practitioners. They also question
how much the mortality rate has declined, suggesting that an apparently
spurious increase in the risk factors may have accounted for most of the
total reduction in the statewide risk-adjusted mortality rate.22




20
  New York is expanding its project to include balloon angioplasty.
21
 Edward Hannan and others, “Improving the Outcomes of Coronary Artery Bypass Surgery in New
York State,” Journal of the American Medical Association, Vol. 271, No. 10 (1994), pp. 761 and 766.
22
 Jesse Green and Neil Wintfeld, “Report Cards on Cardiac Surgeons: Assessing New York State’s
Approach,” New England Journal of Medicine, Vol. 332, No. 18 (1995), pp. 1229 and 1232.



Page 14                                                 GAO/HEHS-99-178 Physician Report Cards
                          B-281938




The Usefulness of         The early experience with physician report cards indicates that
Physician and Physician   organizations are able to address some of the methodological challenges
Group Report Cards        in performance measurement and provide some comparative information.
                          However, their ability to accurately report on broad measures of physician
Remains Limited           performance in a useful manner remains limited.

                          First, while the current report cards measure the performance of physician
                          groups that are smaller than health plans, the medical groups may still be
                          too large to make the cards useful to consumers. One medical group that
                          appears in all three California report cards includes nearly 700 physicians.
                          A consumer faced with the task of selecting a physician could question
                          whether having a set of summary statistics covering so many physicians is
                          really any more helpful than having planwide performance measures for
                          thousands of physicians.

                          Next, the quality of the data and the expense of collecting them are also
                          issues. PBGH, BHCAG, and Health Net used surveys to gather data. As with all
                          survey data, they reflect only the views of patients who chose to respond
                          and then record their recollection or perception of the care they received,
                          which may or may not be accurate. To address these issues, steps must be
                          taken to see if there is bias among respondents compared with
                          nonrespondents and to limit questions to those that patients are likely to
                          answer accurately. And while surveys generally cost less than medical
                          record reviews, they are still expensive to conduct. According to one PBGH
                          official, the Physician Value Check Survey costs approximately $15,000 per
                          medical group.

                          PacifiCare’s Quality Index relies more heavily on gathering and analyzing
                          administrative encounter data for its performance scores—a process that
                          is generally less costly than using surveys but that has other limitations.
                          Administrative data reflect how physicians report the services and
                          procedures they provided rather than the patients’ recollection. However,
                          in some cases, the administrative data are not complete. According to
                          PacifiCare’s Medical Director, the plan receives data from physicians on
                          only about 70 percent of their encounters with patients. He added that
                          publishing the Quality Index has dramatically increased the volume of
                          encounter data submissions. Before the Quality Index was published,
                          PacifiCare received information on about 2 million encounters each
                          month; 2 months after its publication, the plan was receiving data on about
                          5 million encounters per month. Despite the increase in the volume of
                          encounter data that medical groups provide, some groups question the
                          completeness and the quality of the raw data.



                          Page 15                                  GAO/HEHS-99-178 Physician Report Cards
B-281938




Third, current report cards do not provide consistent results. For example,
the PacifiCare and Health Net report cards demonstrate some of the
difficulties when different organizations measure groups in different ways.
The two plans use different methods and different data sources: PacifiCare
uses administrative and other internal data sources, and Health Net uses
enrollee satisfaction survey responses. The two plans also base their
performance measures on a subset of a patient population—plan enrollees
rather than all patients in a medical group. Enrollee satisfaction scores
from the two plans were significantly different in some cases. From the
enrollees’ responses, Health Net classified one medical group as
“excellent”—a classification it gave to only about one-fourth of the
medical groups—while PacifiCare classified the same group in the bottom
third. A consumer looking at Health Net’s report card might be more
inclined to select that medical group than a consumer looking at
PacifiCare’s report card, while a consumer who read both report cards
would be confused as to how a single group could get such disparate
ratings from two plans’ enrollees. According to experts we interviewed,
such divergent scores on similar measures lead to skepticism among
physicians and the general public about the usefulness of report cards.

Finally, the differences in the specifications of the measures and
presentation issues may cause additional confusion. While the report
cards measure some of the same aspects of care, their measures of clinical
quality can be defined and reported differently. For example, one plan may
report the percentage of enrollees who were satisfied with a service, while
another might report only the percentage who indicated that they were
very or extremely satisfied. In addition, reporting issues such as the
relative scale a plan uses can accentuate narrow differences among
medical groups. For example, under PacifiCare’s scale, if all plans in the
comparison fall between 85 and 95 percent on a particular measure, the
group performing the service 85 percent of the time could show up in the
bottom 10th percentile, while the group performing the service 95 percent
of the time could be listed as a “best practices” group. Conversely, if all
the groups perform a recommended service less than half the time, some
of them will still be ranked as best practices groups. Such complexity in
interpreting the results can make consumers wary of report cards.




Page 16                                 GAO/HEHS-99-178 Physician Report Cards
                             B-281938




                             While the work of purchasers, plans, and state agencies represents
New Collaborative            progress toward resolving the difficulties with measuring health care
Efforts and Data             quality, further development is needed. Several national groups have been
Standardization May          organized to cooperatively develop standardized approaches to
                             measurement-related issues. In addition, HHS is taking some steps to
Help Meet Some               facilitate better performance measurement. These efforts are in their
Challenges                   infancy, and it will take time to see what, if any, effect they have on
                             measuring physician performance.


Professional Organizations   Developing a commonly accepted, standardized set of performance
Are Beginning to             measures is a critical step in creating a system of performance
Collaborate on               measurement that will allow “apples to apples” comparisons in health
                             care. Some of the organizations we talked to have recently joined together
Improvements to Physician    to address participant concerns about performance information and the
Report Cards                 factors in the marketplace that impede the flow of data. It remains to be
                             seen whether these coalitions can forge agreement on critical issues that
                             must be addressed in the long term. For these organizations to be
                             successful, disparate groups will have to reach consensus on a number of
                             issues and that will take time.

                             The California Information Exchange is a partnership of purchasers,
                             providers, and other organizations established to promote and protect the
                             exchange of data among health care partners such as health plans,
                             purchasers, and providers.23 According to one Exchange official, the group
                             was formed, in part, to overcome political impediments to the exchange of
                             health care information. The Exchange has formed working groups to
                             develop agreements to be used to define the content, proper use, and
                             format for enrollment data; provider and provider group identifiers;
                             laboratory and encounter records; individual patient identifiers; member
                             identification cards; eligibility data; and pharmacy records. To date, the
                             Exchange’s work groups have adopted agreements or rules of exchange
                             for enrollment data, encounter data, eligibility data, member identification
                             cards, and pharmacy records. The Exchange plans to test these
                             agreements in a series of pilot projects.

                             The Performance Measurement Coordinating Council is sponsored by
                             three health accreditation agencies: the American Medical Association’s
                             American Medical Accreditation Program, the Joint Commission on
                             Accreditation of Health Care Organizations, and NCQA. Comprising 15

                             23
                               Exchange partners include the American Medical Group Association, the California Association of
                             Health Plans, the California Healthcare Association, the California Medical Association, the National
                             IPA Coalition, and the Pacific Business Group on Health.



                             Page 17                                                 GAO/HEHS-99-178 Physician Report Cards
B-281938




members chosen by the founding organizations, it was created in May 1998
to develop efficient and consistent performance measures for different
levels in the health care system. The Council brings together organizations
working on quality measurement in different areas of the health care
industry with different points of view on attribution, the public reporting
of performance data, and the like. For example, the American Medical
Accreditation Program comes from an organization dedicated to
representing the interests of physicians and is most cautious about
attributing performance data to individual physicians and reporting on
performance to the public. At the same time, NCQA, which is a health plan
accreditation organization, is a strong advocate for the public reporting of
data. The progress of the Council illustrates the time it can take to work
on performance measures. For example, it took the Council 8 months to
progress from its formation to the announcement that it intended to
develop a common measurement agenda and to address a range of
performance measurement issues. As of May 1999, one year after its
formation, the council had identified and started work on diabetes care
measures—the first of its measurement sets.

The National Forum for Health Care Quality Measurement and Reporting
is a private, nonprofit entity whose purpose is to develop a comprehensive
quality measurement and public reporting strategy. The Forum followed
from the recommendations of the President’s Advisory Commission on
Consumer Protection and Quality in the Health Care Industry.24 Goals for
the Forum include allowing meaningful quality comparisons of health care
providers and plans and promoting competition in the quality of health
care services. In March 1999, the Forum planning committee approved the
initial members of its board of directors. Representation on the Forum’s
board is broad, including academic researchers and representatives from
AHCPR; HCFA; representatives from consumer, public, and private
purchasers; providers and plans; and research and quality improvement
councils. As with the Performance Measurement Coordinating Council,
the Forum’s efforts are taking time. The Forum took approximately 9
months to select its board of directors and does not expect individual
work groups to begin work until early 2000.




24
 Advisory Commission on Consumer Protection and Quality in the Health Care Industry, Quality First:
Better Health Care for All Americans, Report to the President of the United States (Washington D.C.:
1998).



Page 18                                               GAO/HEHS-99-178 Physician Report Cards
                            B-281938




HHS Is Taking Steps to      HHS has been involved in performance measurement through its
Report on Medicare          administration of the Medicare program, efforts to standardize health data,
Physicians’ Performance     and support of research. In addition to meeting the information needs of
                            Medicare beneficiaries, these efforts can have a substantial effect on
and to Standardize Health   report cards generated by private purchasers and plans. Because most of
Data                        HHS’ initiatives are still in progress, this effect has yet to be determined.


                            The Balanced Budget Act of 1997 (BBA) requires that HHS provide
                            comparative data to Medicare beneficiaries including information about
                            the benefits, quality, and performance (to the extent available) of health
                            care options in their area to assist them in making informed choices under
                            the Medicare+Choice Program.25 To provide better quality and
                            performance comparisons, HCFA contracted with Health Economics
                            Research Inc. in September 1997 for assistance in developing a
                            performance measurement system. The contractor is studying the
                            feasibility of using HEDIS measures for fee-for-service Medicare at the
                            group practice, local, and national levels. As part of this 3-1/2-year
                            contract, five clinical measures relevant to the Medicare
                            population—retinal eye examinations for diabetics, follow-up care after
                            mental health hospitalization, breast cancer screening with
                            mammography, beta blocker treatment after a heart attack, and the Health
                            of Seniors survey results—are being examined at four large group
                            practices.

                            Although the study on HEDIS measures for group practices is not expected
                            to be completed until 2001, some difficulties, such as those associated
                            with small sample sizes, have already been identified.26 For example, while
                            the four group practices each had between 4,000 and 40,000 Medicare
                            fee-for-service beneficiaries, sample sizes for each measure fell
                            considerably once population subsets of gender, age, or condition were
                            identified. HEDIS specifies 411 patients as a sufficient sample size, but this
                            was obtained only for two of the three claims-based measures—breast
                            cancer screening and retinal eye examinations. For these two measures,

                            25
                             Created by the BBA, the Medicare+Choice program is designed to allow beneficiaries to choose
                            health care from Medicare’s traditional fee-for-service program and a range of health plans, such as
                            health maintenance organizations and provider-sponsored organizations, participating in Medicare.
                            26
                              In addition to the challenge of developing comparable performance information for group practices
                            under fee-for-service Medicare, HCFA will have to ensure that the information provided to
                            beneficiaries is clear, sufficient, and helpful to their decisionmaking or it will not be used. For
                            example, we previously reported on problems with HCFA’s efforts to provide comparative information
                            on health plans—HCFA had not provided information that was easy for beneficiaries to understand.
                            See Medicare: HCFA Should Release Data to Aid Consumers, Prompt Better HMO Performance
                            (GAO/HEHS-97-23, Oct. 22, 1996) and Medicare: Progress to Date in Implementing Certain Major
                            Balanced Budget Act Reforms (GAO/T-HEHS-99-87, Mar. 17, 1999). Physician-level information runs
                            the risk of having similar problems.



                            Page 19                                                 GAO/HEHS-99-178 Physician Report Cards
B-281938




the sample size was large enough only for the three group practices with
more than 20,000 Medicare fee-for-service beneficiaries.

HHS’ efforts to establish standards for information transactions and data
elements, including unique identifiers for individuals, plans, and providers,
may also have an effect on performance measurement systems. Under the
Health Insurance Portability and Accountability Act of 1996 (HIPAA) (P.L.
104-191), HHS is required to adopt standards to support the exchange of
information on administrative and financial health care transactions.27 The
standards are to include data elements and code sets for the electronic
exchange of information; unique health identifiers for health care
providers, health plans, employers, and individuals for use in the health
care system; and security protections against the unauthorized disclosure
and use of health information. The standards are to apply to all health
plans, health care clearinghouses, and health care providers that transmit
health information electronically.28

Some standards under development, such as the unique identifier for
individuals, have been contentious. In response to concerns about privacy,
three bills were introduced in the 105th and 106th Congresses to repeal the
requirement for HHS to adopt a standard unique health identifier. While the
two bills from the 105th Congress expired and one is pending before the
current Congress, the Omnibus Consolidated and Emergency
Supplemental Appropriations Act for 1999 (P.L. 105-277) provided that no
funds available under the act be used to adopt a final standard for
individual unique health identifiers until legislation is enacted specifically
approving the standard. While the implementation of the HIPAA standards
has the potential to significantly improve the usability of the health data
available for performance measurement, it will not address all the data
challenges, such as those related to data accuracy.

An additional HHS effort under way to further physician performance
measurement is the development of a consumer satisfaction survey for
fee-for-service Medicare beneficiaries. AHCPR officials said that they are
interested in adapting the CAHPS survey—an instrument for measuring
consumer satisfaction and experience with health plans—to the provider

27
 Transactions include health claims or equivalent encounter information, enrollment and
disenrollment in a health plan, health care payment and remittance advice, health plan premium
payments, first report of injury, health claims status, and referral certification and authorization.
28
  Under HIPAA, standards were required to be enacted by August 21, 1999, regarding the privacy of
individually identifiable health information that is electronically exchanged. Because this deadline was
not met, HIPAA now requires the Secretary of HHS to establish standards by regulation no later than
February 21, 2000.



Page 20                                                   GAO/HEHS-99-178 Physician Report Cards
                  B-281938




                  level. AHCPR is studying the use of the CAHPS survey with smaller units,
                  such as group practices or individual physicians. In addition, AHCPR is
                  sponsoring research on performance measurement and is working with
                  others to develop a framework for measuring health care performance.


                  Consumers could use more information on the quality of health care
Conclusions       providers to help them make informed choices about where to seek care.
                  Comparative information on physicians is important to all consumers,
                  whether they enroll in traditional Medicare or in a private health plan or
                  face a choice of primary care physicians when they join a managed care
                  plan. Yet the field of physician performance measurement is still in its
                  infancy. Challenges to developing physician report cards include selecting
                  performance measures that satisfy the information needs of various
                  audiences and gaining the cooperation of physicians and consumers
                  required to assemble consistent and credible performance data. The
                  experience of several organizations in producing medical group or
                  specialty care report cards indicates that steps can be taken to better
                  inform consumers, but the challenges that remain limit the report cards’
                  usefulness. Given sufficient time, public and private efforts to develop a
                  consensus on standardized data collection and comparable quality
                  measurement may lead to more useful measures for consumers through a
                  more efficient system for providers and plans nationwide.


                  We obtained informal comments on a draft of this report from the Senior
Agency Comments   Clinical Adviser in HCFA’s Office of Clinical Standards and Quality. She
                  agreed with the report’s general findings. She suggested that under the
                  fee-for-service payment system, we highlight the problem of determining
                  which physician is accountable for managing a patient’s care. The logistics
                  of establishing the linkages by means of existing medical records is
                  another area of concern that she recommended we stress in the report.

                  We also obtained comments on the draft from an expert in quality
                  measurement who suggested that we include more information on the
                  methodological challenges of assessing physician performance. He felt
                  that a stronger critique of the validity of currently available measures
                  would be helpful in the analysis of physician report cards. He noted that it
                  is impossible to differentiate among providers with current physician
                  report cards and warned against the dangers of misinformation. He
                  encouraged us to place more emphasis on the need for research efforts to
                  develop better measures that provide valid information and to improve our



                  Page 21                                  GAO/HEHS-99-178 Physician Report Cards
B-281938




understanding of preferred clinical strategies. He also emphasized the
need for developing electronic medical records for access to clinically
relevant data.


As agreed with your office, unless you publicly announce its contents
earlier, we plan no further distribution of this report until 30 days from the
date of this letter. We will then send copies of this report to the Secretary
of HHS and others who are interested. We will also make copies available to
others on request.

The information contained in this report was developed by Rosamond
Katz, Assistant Director, Mark Ulanowicz, and Patricia K. Yamane. Please
contact me at (202) 512-7114 or Rosamond Katz at (202) 512-7148 if you or
your staff have any questions.

Sincerely yours,




Janet Heinrich
Associate Director, Health Financing
  and Public Health Issues




Page 22                                   GAO/HEHS-99-178 Physician Report Cards
Page 23   GAO/HEHS-99-178 Physician Report Cards
Contents



Letter                                                                                          1


Appendix                                                                                       26
                    Pacific Business Group on Health                                           26
Report Cards        PacifiCare                                                                 28
Comparing Medical   Health Net                                                                 29
                    Minnesota’s Buyers Health Care Action Group                                33
Groups
Figures             Figure I.1: 1996 PBGH Physician Value Check Survey Scores for              27
                      Three Southern California Medical Groups
                    Figure I.2: 1998 PacifiCare Quality Index Scores for Its                   28
                      Commercial Members at Three Medical Groups
                    Figure I.3: 1998 Health Net Participating Physician Group Report           30
                      Care
                    Figure I.4: 1998 Health Net Asthma Report Card                             32
                    Figure I.5: 1998 BHCAG Report Card on One Care System                      33




                    Abbreviations

                    AHCPR      Agency for Health Care Policy and Research
                    BBA        Balanced Budget Act of 1997
                    BHCAG      Buyers Health Care Action Group
                    CABG       coronary artery bypass graft
                    HCFA       Health Care Financing Administration
                    HEDIS      Health Plan Employer Data and Information Set
                    HHS        Department of Health and Human Services
                    HIPAA      Health Insurance Portability and Accountability Act of 1996
                    HMO        health maintenance organization
                    IPA        independent practice association
                    NCQA       National Committee on Quality Assurance
                    PBGH       Pacific Business Group on Health
                    PPO        preferred provider organization


                    Page 24                                 GAO/HEHS-99-178 Physician Report Cards
Page 25   GAO/HEHS-99-178 Physician Report Cards
Appendix

Report Cards Comparing Medical Groups


                   In California, a purchasing group, the Pacific Business Group on Health
                   (PBGH), and two health plans, PacifiCare and Health Net, have moved a
                   step closer to reporting on physician performance by publishing report
                   cards on medical groups and independent practice associations (IPA). In
                   addition, another purchasing group, the Buyers Health Care Action Group
                   (BHCAG) in Minnesota, has published report cards on care systems, which
                   can be similar to large medical groups. Below, we illustrate the
                   information generated on 3 of the 11 medical groups common to all three
                   California report cards and one Minnesota care system. In California,
                   medical group A includes more than 100 physicians, medical group B has
                   675 physicians, and medical group C has more than 300 physicians. In
                   Minnesota, the care system includes 550 physicians.


                   PBGH  publishes a report card on medical groups that is based on its
Pacific Business   Physician Value Check Survey. Figure I.1 shows the scores for three
Group on Health    southern California medical groups as reported on the PBGH Internet site.
                   The numerical scores are divided into three categories that indicate
                   relative measures of performance: above average, below average, and
                   average.




                   Page 26                                 GAO/HEHS-99-178 Physician Report Cards
                                         Appendix
                                         Report Cards Comparing Medical Groups




Figure I.1: 1996 PBGH Physician Value Check Survey Scores for Three Southern California Medical Groups




                                         Note: The numerical scores represent scores based on survey responses for the medical groups.
                                         Scores above and below average indicate a relative measure of the groups.

                                         Source: Pacific Business Group on Health, California Consumer Healthscope, at
                                         http://www.healthscope.org.




                                         Page 27                                             GAO/HEHS-99-178 Physician Report Cards
                                          Appendix
                                          Report Cards Comparing Medical Groups




                                          A consumer reading this report card would learn that medical group B
                                          received high overall scores for satisfaction and cholesterol screening.
                                          However, a person with high blood pressure would notice that while
                                          groups B and C scored below average in prescribing medicine for high
                                          blood pressure, group C had better success reducing its patients’ blood
                                          pressure. People with diabetes would not find any information specific to
                                          the treatment of their condition.


                                          PacifiCare’s Quality Index report card reflects the health care experiences
PacifiCare                                and opinions of members of the PacifiCare health plan only. Figure I.2
                                          shows the 1998 Quality Index scores for the same medical groups
                                          highlighted in figure I.1. The numerical scores represent a percentile rank
                                          for a medical group compared with that of all other groups. PacifiCare
                                          identifies groups as best practice groups for a particular measure if they
                                          are in the top 10 percent relative to other groups. These are denoted by a
                                          diamond next to the number in the table.



Figure I.2: 1998 PacifiCare Quality Index Scores for Its Commercial Members at Three Medical Groups




                                          a
                                           Data below threshold: the medical group did not have enough PacifiCare enrollees with
                                          congestive heart failure to allow for statistically valid measurement.
                                          b
                                           Includes responses from both commercial and Secure Horizons members.




                                          Page 28                                              GAO/HEHS-99-178 Physician Report Cards
             Source: PacifiCare.


             The comparative performance information in figure I.2 is limited and
             selective and may not be adequate for choosing a medical group. A
             consumer reading this report card would learn that PacifiCare members
             using medical group B were not happy with their access to care relative to
             the other groups’ patients—it scored in the bottom 15 percent for
             access-related complaints—but were very satisfied with the group’s
             primary care physicians—rating them in the top 10 percent in satisfaction.
             The report card also indicates that medical group C was in the bottom
             third of medical groups for cervical cancer screening but in the top
             10 percent for benefits appeals to PacifiCare.


             Health Net’s Participating Physician Group Report Card is derived from a
Health Net   satisfaction survey of the plan’s members. Figure I.3 shows selected
             Health Net report card scores for the same three southern California
             medical groups as shown in figures I.1 and I.2. The numerical scores
             represent the percentage of respondents who reported that they were
             satisfied regarding each measure, and the groups are classified as
             excellent, very good, or good.




             Page 29                                 GAO/HEHS-99-178 Physician Report Cards
                                           Appendix
                                           Report Cards Comparing Medical Groups




Figure I.3: 1998 Health Net Participating Physician Group Report Care




                                           Source: Health Net Participating Physician Group Report Card, Sept. 1998.




                                           A consumer reading this report card would find that the three medical
                                           groups were largely undifferentiated. They all were rated either very good
                                           or excellent, both overall and within the three broad categories of quality
                                           of care, access to care, and medical group satisfaction. Unlike the
                                           PacifiCare Quality Index report, the Health Net participating physician
                                           group report card provides information only on members’ satisfaction with
                                           each issue—it does not provide information on the extent to which




                                           Page 30                                              GAO/HEHS-99-178 Physician Report Cards
Appendix
Report Cards Comparing Medical Groups




particular services, such as mammograms or cervical cancer screenings,
were provided.

Health Net is also developing report cards on care provided by medical
groups for certain chronic conditions. Figure I.4 shows the asthma report
card scores for medical groups A and B. Medical group C did not have
enough asthma patients responding to the survey to be included in the
comparison. The four stars denote “very good.”




Page 31                                 GAO/HEHS-99-178 Physician Report Cards
                                          Appendix
                                          Report Cards Comparing Medical Groups




Figure I.4: 1998 Health Net Asthma Report Card




                                          Source: Health Net Participating Physician Group Asthma Care Report Card, Dec. 1998.




                                          For patients with asthma trying to choose a medical group, the Health Net
                                          asthma report card provides a considerable amount of information on
                                          clinical quality, including information on outcomes of care. It indicates
                                          that even though both groups were rated very good for asthma care, no
                                          patients with asthma in group A and fewer than 1 in 10 in group B reported




                                          Page 32                                             GAO/HEHS-99-178 Physician Report Cards
                                        Appendix
                                        Report Cards Comparing Medical Groups




                                        using a peak-flow meter daily, even though daily use is recommended in
                                        national clinical guidelines. In addition, the report tells consumers that a
                                        higher share of the survey respondents from medical group A reported no
                                        asthma-related absences from work or school in the past month than
                                        respondents from medical group B.


                                        BHCAG  is currently reporting on 12 measures focusing on such issues as
Minnesota’s Buyers                      access to services and interactions with physicians. Figure I.5 shows the
Health Care Action                      results of the BHCAG survey for one care system, a 550-physician
Group                                   multispecialty medical group. The comparison rating shows whether the
                                        survey ratings for the care system are better than, similar to, or below the
                                        average rating. The numerical scores are the statistics for each measure.



Figure I.5: 1998 BHCAG Report Card on One Care System




                                        Source: Choice Plus 1999 Consumer Satisfaction Survey Results.




                                        Page 33                                             GAO/HEHS-99-178 Physician Report Cards
           Appendix
           Report Cards Comparing Medical Groups




           A consumer reading this report card would learn that this care system was
           scored average by its patients in terms of overall satisfaction and quality of
           care but was scored below average in the areas related to interaction with
           physicians, such as a physician’s explaining medical procedures and tests.




(108386)   Page 34                                   GAO/HEHS-99-178 Physician Report Cards
Ordering Information

The first copy of each GAO report and testimony is free.
Additional copies are $2 each. Orders should be sent to the
following address, accompanied by a check or money order
made out to the Superintendent of Documents, when
necessary. VISA and MasterCard credit cards are accepted, also.
Orders for 100 or more copies to be mailed to a single address
are discounted 25 percent.

Orders by mail:

U.S. General Accounting Office
P.O. Box 37050
Washington, DC 20013

or visit:

Room 1100
700 4th St. NW (corner of 4th and G Sts. NW)
U.S. General Accounting Office
Washington, DC

Orders may also be placed by calling (202) 512-6000
or by using fax number (202) 512-6061, or TDD (202) 512-2537.

Each day, GAO issues a list of newly available reports and
testimony. To receive facsimile copies of the daily list or any
list from the past 30 days, please call (202) 512-6000 using a
touchtone phone. A recorded menu will provide information on
how to obtain these lists.

For information on how to access GAO reports on the INTERNET,
send an e-mail message with "info" in the body to:

info@www.gao.gov

or visit GAO’s World Wide Web Home Page at:

http://www.gao.gov




PRINTED ON    RECYCLED PAPER
United States                       Bulk Rate
General Accounting Office      Postage & Fees Paid
Washington, D.C. 20548-0001           GAO
                                 Permit No. G100
Official Business
Penalty for Private Use $300

Address Correction Requested