Chapter 1. Introduction
1.1. Background
Public recognition of health care quality issues has spiked remarkably in
the past ten years, driven by a series of high-profile reports from the Institute
of Medicine (IOM), the RAND Corporation, and others.6-9 These
reports showed, among other facts, that preventable medical errors in hospitals
result in as many as 98,000 deaths per year; preventable medication errors
occur at least 1.5 million times per year; and on average, only 55% percent
of recommended care is delivered.
In response, a variety of stakeholders from across the spectrum of health
care delivery—including providers, professional and hospital associations,
accreditation organizations, employers and business groups, insurance companies,
and state and federal governments—have focused on monitoring and improving
the quality of care. A primary goal has been to avoid unnecessary deaths
and poor health—at a minimum, to "first do no harm" by
limiting the amount of injury incurred by health care itself. Another
strong motivation has been the increasing recognition that value for money
is poor in the health care system. Health care costs are high and have
been increasing rapidly, without the level or improvement in quality that is
expected by those paying the costs.
In this environment, there is widespread agreement that "business as usual"—i.e.,
exclusively relying on providers to self-police the quality of care they provide
without systematic measurement, monitoring, and reporting—is no longer
adequate for improving the quality of care.10
Quality
is increasingly recognized as a product of systems, not individuals, and attention
has focused on how to analyze and redesign those systems. A consensus has
emerged that although solutions will not be easy, "a key component of any
solution... is the routine availability of information on performance
at all levels"11—that
is, "quality of care should be measured and reported routinely at both
the national and provider-specific (e.g., hospital and physician) levels."12 More
simply, "that which cannot be measured is difficult to improve."13
There are a variety of measurement-based approaches to improving quality. The
most widespread and longest-used approach, referred to in this report as "quality
improvement," is an internal effort by providers or other organizations
to measure quality of care, identify areas of weakness, devise and implement
changes, and monitor the effects of the changes. As results are typically not
shared outside of the organization that conducts the quality improvement project,
this approach can be viewed as formalization of the traditionally prevailing
model of improving quality—mutual peer review. More recently, calls
for external accountability of health care providers have led to two types
of quality monitoring approaches. One is public reporting of quality of health
care providers.b
These
reports can steer patients to higher-quality care and drive providers to improve
their scores in order to bolster their professional reputation. The other one
is linking quality of care to financial incentives, either in the form of pay-for-performance
(paying more for good performance on quality metrics) or in the form of tiered
insurance products, which steer patients towards higher-quality providers by
charging higher copayments for visits to providers with poorer quality scores.
These efforts all depend on reliable, valid, and agreed-upon ways to measure
the quality of care. However, health care quality is difficult to measure,
in particular because there are many factors contributing to outcomes of health
care in addition to the actual quality of the care provided. Many of
these factors—for example, patient compliance with treatment—are
largely out of providers' control. Despite this and other difficulties,
much progress has been made in the science of quality measurement. The
underlying knowledge base has greatly expanded. The science of measuring
health status has improved, as has the evidence supporting "best practices" that
have been proven to lead to improvements in health status. This evidence
base has allowed for the development of numerous quality indicators, which
then have been tested for reliability, validity, ease of use, and usefulness
for improving quality.
The quality indicators fall into two main categories:
counts of inputs or actions that are known to lead to better health outcomes
("structure" and "process" measures in Donabedian's
classic quality measurement framework),14 and
direct measurements of the outcomes of care ("outcomes"). Unfortunately,
many of the most rigorous measures are also the most difficult to implement
due to their burden of data collection. The data needed for quality measurement
must be collected specifically for that purpose, abstracted from (usually)
paper medical records, or abstracted from existing electronic data that have
been collected for other purposes, notably billing. However, as will
be shown in this report, there is now a small but growing arsenal of well-tested
quality indicators that can be used with existing data sources or data collection
processes.
The Agency for Healthcare Research and Quality (AHRQ) was one of the pioneers
of quality indicator development. As early as 1994, its Healthcare Cost
and Utilization Project (HCUP) developed a publicly available set of quality
indicators for hospital care based on discharge data. Congress later
tasked AHRQ with developing an annual report on the quality of health care
in the United States, the National Healthcare Quality Report.15 In
preparation for this report, in 2001 AHRQ invested in an update of the HCUP
indicators, which were renamed the AHRQ Quality Indicators (AHRQ QIs).16 Indicators
of patient safety were added in 2002.17 The
AHRQ QIs and associated computer code are now publicly distributed and supported
by AHRQ, with regular updates.18 They
are widely used by a variety of organizations for many different purposes. Meanwhile,
many other organizations, both public and private, have developed and used
their own sets of quality indicators.
Given the rapid growth of and robust demand for quality indicators, it is
important to assess the position of the AHRQ QIs in the quality indicator "market." Who
is using the AHRQ QIs today, and for what purposes? What have users' experiences
been, and what are their unmet needs? Who else is developing and/or distributing
indicators similar to the AHRQ QIs? Most importantly, what has been the
impact of the AHRQ QIs on the quality of care delivered to patients?
To
understand the answer to these and related questions, the RAND Corporation
was asked to conduct a one-year evaluation to assess user experiences with
the AHRQ QIs, and to identify users of other quality indicators, vendors of
quality measurement products using the AHRQ QIs, and developers of quality
indicators comparable to the AHRQ QIs. The results of this study are intended
to inform decisions by AHRQ on future priorities for the QI program.
Return to Contents
1.2. Description of the AHRQ Quality Indicators (AHRQ QIs)
AHRQ has developed several types of QIs, all of which share certain core characteristics.
There are currently four sets of AHRQ QIs:
- Inpatient Quality Indicators (IQIs). There are 32
IQIs reflecting the quality of care provided in hospitals. The indicators
fall in four categories.
- The first category includes the in-hospital mortality rates for seven
specific medical conditions. The conditions were selected because
their mortality rates were shown to vary across providers and because evidence
indicates that the mortality risk for those conditions may depend on quality
of care. The mortality rates, adjusted for patient risk, can be compared
to those of other providers or to other benchmarks, with lower rates indicating
better quality.
- The second category includes the in-hospital mortality rates for eight
surgical procedures. Like the condition-specific mortality rates,
the procedures were chosen because their mortality risk was known to vary
across providers and because evidence indicates that mortality for those
procedures may be associated with inferior quality of care. The mortality
rates can be benchmarked, and lower rates are better.
- The third category includes utilization rates for eleven procedures for
which there is potential for overuse, underuse, or misuse. Four of the
utilization rates were designed for comparisons between geographic areas
(either counties or Office of Management and Budget Metro Areas), not hospitals.
The rates are measured as the number of procedures performed divided by the
total number of patients who were potential candidates for that procedure.
Utilization rates that are much higher or lower than benchmarks may point to
poor quality. The direction of the indicator—i.e., whether higher or
lower rates are desirable—depends on the condition. For example,
Cesarean delivery has been identified as an overused procedure, so lower rates
indicate better quality (IQI 21). In contrast, vaginal delivery following
a previous Cesarean delivery (VBAC) is considered an underused procedure,
so higher rates indicate better quality (IQI 34).
- The fourth category includes indicators of the hospital-level volume
for six complex procedures for which research suggests a positive impact
of case volume on patient outcomes. The procedure volume therefore
provides an indirect indication of the provider's expected outcome
for the procedure. The indicators are simple counts of the number
of times that the procedures are performed per year. These counts
can be compared to thresholds for the minimum annual volume of procedures
that providers should perform to ensure an acceptable level of quality. The
volume thresholds are based on published research.
- Patient Safety Indicators (PSIs). The PSIs contain
27 indicators of potentially-avoidable complications or other adverse events
related to hospital care. For example, an infection resulting from medical
treatment (PSI 7) is considered an avoidable complication, and the rate of
surgical instruments or other "foreign bodies" left in the patient
following surgery (PSI 5) is considered a preventable adverse event. The
indicators are measured as the number of complications or adverse events
divided by the total number of patients at risk for those events. Twenty
of the indicators were designed for comparisons between hospitals, and the
remaining 7 were designed for comparisons between geographic areas (either
counties or Office of Management and Budget Metro Areas).
- Prevention Quality Indicators (PQIs). The PQIs are
based on the hypothesis that appropriate ambulatory care can help to
prevent hospital admissions for some conditions. The indicators consist
of hospital admission rates for 14 of these ambulatory care-sensitive conditions
and are intended for comparisons of geographic areas. Unlike the other
AHRQ QIs, they do not reflect quality of hospital care, but the quality of
ambulatory care. The indicators are measured as condition-specific admission
rates (e.g., the number of admissions for urinary tract infection per 100,000
people living in a Metro Area or county). Each admission to the hospital
reflects a failure in the delivery of appropriate care outside the hospital,
so lower rates reflect better quality.
- Pediatric Quality Indicators (PDIs). The PDIs
contain 18 indicators, each of which is similar to a comparable type of IQI,
PQI, or PSI, except that the PDIs in each case refer to the pediatric population. Since
these indicators were released during the course of this study, they were
not included in the analysis. Several of the PDIs were formerly included
in the AHRQ IQI, PSI, and PQI sets. Most of the issues raised in this
report would generalize to the PDIs.
There are several key characteristics that apply to all four sets of the AHRQ
QIs.
- Based on administrative data.The AHRQ QIs are based
on hospital discharge data that are collected using the UB-92 standardized
claim reporting form and used mainly for billing purposes. The indicators
can be used with existing hospital administrative databases, including
the HCUP National Inpatient Sample and Medicare MedPAR data.
- Outcome-focused. Most of the AHRQ QIs reflect health
care outcomes, and not the rates at which evidence-based processes of care
are followed. The exceptions are the subset of IQIs which reflect
utilization or volume of services.
- Hospital-focused. Most of the indicators focus
on inpatient care, and all of the indicators are based on hospital data.
The PQIs reflect the quality of ambulatory care, but the measurement focuses
on admissions to the hospital that could have been prevented given appropriate,
effective ambulatory care.
- Reviewed for scientific soundness. The AHRQ QIs
were tested by the Evidence-Based Practice Center at the University of
California San Francisco and Stanford University. Detailed documentation
of technical information, including the methodology, evidence base, and
measurement properties of the indicators, is available in the public domain.
- Available for public use. The AHRQ QIs and associated
code for SAS, SPSS,c and
Windows are available for public download at no cost to the user. User
support is also provided free of charge.
The AHRQ QIs were originally designed to be used for research and quality
improvement activities.19 However,
as examples included in this report demonstrate, the QIs are now used for a
variety of other purposes, including public reporting, pay-for-performance,
and segmenting providers for tiered insurance products. AHRQ has evaluated
and endorsed the use of the QIs for these new purposes, given certain caveats.20
Return to Contents
1.3. Objectives
This report has three main objectives:
- Provide an overview of the market for the AHRQ QIs as well as indicators
and quality measurement tools developed by other organizations that are
similar to the AHRQ QIs or that incorporate the AHRQ QIs.
- Provide an overview of the range of ways in which the AHRQ QIs are used
by various organizations.
- Assess the market demand for the AHRQ QIs, identify unmet needs, and
discuss implications for future activities by AHRQ.
Return to Contents
1.4. Organization of the Report
The remainder of this report is divided into six sections. Section
2 outlines the methodology used in our evaluation. Section 3 assesses
the market for quality indicators and AHRQ's role in that market. Section
4 evaluates the AHRQ QI program on the dimensions of importance, scientific
soundness, usability, and feasibility. Section 5 presents two case studies
of how the AHRQ QIs are used in particular market areas. Section 6 summarizes
lessons learned, and Section 7 concludes with a discussion of the implications
of the evaluation for the future of the AHRQ QI program.
b.Quality comparisons
have been made between states, health plans, nations, etc. in addition to between
providers. We focus on provider comparisons here since they are the most
common.
c.Support for SPSS
will be discontinued by AHRQ in 2007.
Return to Contents
Proceed to Next Section
540 Gaither Road Rockville, MD 20850