The information in this report is presented in numerous formats. We have included more information than is available on the Hospital Compare web site - all of which came from data collected by CMS. To allow for more options for comparison, for example, this report allows consumers to look at all of a state's hospitals at once, while the Hospital Compare web site only allows comparison of scores of three hospitals at a time. This report also includes historical data, which reveals hospitals' progress over time - at both state and individual hospital levels. Comparisons using hospitals' compliance averages within each state can also be viewed on the National Compliance pages. Finally, the report allows for sorting the information by hospital (alphabetical), percentage of compliance, and the change between the most current compliance and the quarter immediately preceding it. Graphics show the change over time for each hospital and for each measure in each state.

How to find your hospital

There are two ways to find your hospital's compliance rates. On the landing page of the report, type your hospital's name in the space provided. Do not use any punctuation marks. You can also click on your state on the US map, choose a measure, select the "All" tab, find your hospital in the alphabetical listing; and click on the hospital name to find a complete summary of your hospital's record on complying with all three measures over time.

Missing hospitals

If your hospital does not appear on the site, it could be included under another hospital's name. Some hospitals have multiple facilities that bill under a single provider number and may report measures jointly. CMS collects this data on preventing hospital-acquired infections based on provider numbers. Therefore, the names of some hospitals that report data will not appear in this report. The CMS data used for this report does not indicate details regarding which facilities jointly report, so there is no way for us to know which hospitals jointly report.

Why some hospitals don't report

Payments to Critical Access Hospitals are not affected by reporting, because they are not eligible for the incentive payment.

However, according to CMS, 42% of these hospitals do report.ⁱ CMS defines these as

"A small, generally geographically remote facility that provides outpatient and inpatient hospital services to people in rural areas. The designation was established by law, for special payments under the Medicare program. To be designated as a CAH, a hospital must be located in a rural area, provide 24-hour emergency services; have an average length-of-stay for its patients of 96 hours or less; be located more than 35 miles (or more than 15 miles in areas with mountainous terrain) from the nearest hospital or be designated by its State as a 'necessary provider.' Hospitals may have no more than 25 beds."

Centers for Disease Control and Prevention (CDC) information on preventing surgical site infections

• CDC's Frequently Asked Questions about surgical site infections (SSI):
  http://www.cdc.gov/ncidod/dhqp/FAQ_SSI.html
• CDC complete guidelines for avoiding SSI, last updated in 1999. These guidelines also include information on the selection of appropriate antibiotics.
  http://www.cdc.gov/ncidod/dhqp/gl_surgicalsite.html
• CDC Surgical Infection page:
  http://www.cdc.gov/ncidod/dhqp/dpac_ssi.html
• Details on how SCIP data is submitted
  http://manual.jointcommission.org/bin/view/Manual/SurgicalCareImprovementProject

Types of surgeries for which the measures are reported

Hospitals do not report on all surgeries, rather they report on a subset of operations specified by CMS. The SCIP measures included in this report are based on scientific evidence that only looked at certain surgeries. Thus, the data reported here by hospitals cover a subset of all operations performed at a hospital: colon surgery, hip and knee arthroplasty, abdominal and vaginal hysterectomy, cardiac surgery (including coronary artery bypass grafts or CABG) and vascular surgery. For more details, see Hospital Compare's Information for Professionals, Process of Care measures tab, scroll down to the link to measures list.

Exceptions for recommended care

Patients who should not get the recommended care because of a specific medical situation, such as an allergy to a particular medication, are not included in the calculation of these measures. The compliance rates include only patients whose histories and conditions indicate the treatment is appropriate.

Process measures

The activity being measured in this report is a "process measure." That is, rather than giving the outcomes of care (death, infection or a rate of these outcomes), process measures give the rate at which a health care provider gives recommended evidence-based care. Typically these process measures have been studied for years and are well-documented as a standard of care to improve health care outcomes.

High and Low compliance levels

Consumers Union set compliance levels based on our analysis of the data. We defined a hospital to be in high compliance if it administered a procedure to at least 95% of its eligible patients. The U.S. Department of Health and Human Services recently released a "National Action Plan to Prevent Healthcare-Associated Infections" that set a target of 95% compliance with SCIP measures within the next five years. We believe that is too long to wait for hospitals to meet this goal considering that use of antibiotics to prevent surgical site infections "is one of the most widely accepted practices in surgery"ⁱⁱ and the evidence of the effectiveness of the SCIP measures was published in 2005, after which CMS began collecting data on hospitals' compliance. Further, CMS, in its Information to Consumers reports that "the goal for each measure is 100 percent."

We chose to label hospitals with rates of 79% and below as low compliance because these hospitals are putting at least 20 percent of their patients at a higher risk for developing surgical infections. Even with as few as 10 reported surgeries, a compliance rate of 79% or less is significantly lower than the upper threshold of 95%, with a very high degree of statistical confidence.

Information about the Hospital Compare Surgical Care Improvement Project (SCIP)

When payments were tied to reporting

The initial years of reporting measures to CMS were totally voluntary, with no penalties for hospitals that didn't report. Consequently, in those years, the number of hospitals reporting was small. The first two quarters of data provided in this report (September 2005 and December 2005) reveal that the majority of hospitals reported no data ("NA"). But in July 2006, CMS began tying Medicare payments to reporting the SCIP data. Hospitals that failed to comply with reporting lost 2% of their Medicare annual payment update. In the December 2006 quarter, the number of hospitals reporting increased significantly, demonstrating that voluntary reporting systems fail to give the public a complete picture and that connecting payments to reporting is a powerful tool.

How many hospitals report

Almost all hospitals report these measures. According to Benedicta Able-Steinberg with CMS, 3800 acute care hospitals currently report. According to Dr. Dale Bratzler, in the last quarter of 2007, 98% or 3750 acute care hospitals reported. According to Dr. Dale Bratzler, 800-900 of US acute care hospitals do not do surgery.ⁱⁱⁱ

Why SCIP 2 has limited information

Hospitals began reporting SCIP 2, "the right drug" measure, at the same time as the others, but the early data were not released to the public. Thus the SCIP 2 history only goes back to June 2007.

Sampling used by CMS

Whether or not a hospital uses sampling when reporting compliance data is determined by rules established by The Joint Commission and CMS. Hospitals are not required to sample. See Instructions to hospitals on how to sample; each hospital is responsible to use the appropriate techniques with oversight from their performance measurement system. The Joint Commission requires accredited hospitals to participate in an acceptable performance measurement system. According to the CMS instructions: "The sample cases should be randomly selected in such a way that the individual cases in the population have an equal chance of being selected. Only when the sample data truly represent the whole population can the sample-based performance measure data be meaningful and useful...The Surgical Care Improvement Project (SCIP) sample is done by sub-population. For measures requiring medical record abstraction, sampling must be done using available databases that contain all discharges for the transmission quarter."

On the "Cutting Surgical Infections" website's state detail pages for each measure, the "notes" column indicates which hospitals used sampling. These notes accompanied the data that Consumers Union downloaded from the CMS Hospital Compare website. In numerous hospital cases, the notes indicate that both sampling was done and the number of cases reported by the hospital was small. This does not appear to comply with the process of sampling for hospitals with a larger volume of surgical patients. As CMS sampling specifications indicate: "Sampling should not be used unless the hospital has a large number of cases in the Initial Patient Population because a fairly large number of sample cases are needed to achieve a representative sample of the population." It appears that most of these dual notations involve small hospitals that are doing few surgeries, but there is no way for us to know the actual numbers of surgical patients from which these samples were taken.

Further, according to Dr. Dale Bratzler, some modifications have been made to the sampling protocols (changes in the procedure codes from which the sample is drawn) and the specific surgeries reported which could account for the decreases in numbers of patients in some of the charts.

Patient volume

Using patient volume in addition to percent of hospitals complying

CMS uses the average of hospital compliance rates in calculating the average rate of compliance for a state. This is called the "state average." This skews results as it gives low volume hospitals as much weight in this calculation as larger hospitals. While this report allows for comparisons of the states using similar averages, our primary comparison is done using weighted averages of the hospital compliance rates, with hospital rates weighted by patient sample size. This approach gives a more accurate estimate of the percentage of patients in a state who received the right care. This approach would produce more accurate estimates if actual patient volumes were used in the calculation; however, CMS does not make patient volumes available. Instead, we used the only sample size available for weighting hospitals in the calculation of state averages, which is the number of patients sampled within each hospital. The website allows for sorting the states by both - "patient average" (weighted) and "hospital average" (unweighted).

Hospitals with low patient volume

CMS issues a disclaimer for hospitals reporting on fewer than 25 patients, concluding that the scores for these hospitals do not reliably predict hospital performance because the number of patients is too small. On the Hospital Compare site, CMS omits scores for hospitals caring for a low number of patients. However, we chose to include these patients in calculating the weighted statewide average to give the public a complete picture of states' records on implementing these life-saving prevention strategies. CMS concludes that these practices should be used for all surgery patients. Even if a hospital only performs a small number of operations, the expectation that each of its few patients gets this preventive treatment is the same as for a hospital performing hundreds of patients. The bottom line is that every surgical patient should expect these preventive measures to be taken, whether in a small or large hospital.

However, reports on individual hospitals with small samples of surgical patients should be viewed with caution. Consumers may mistakenly think that the performance of a small volume hospital with high compliance is comparable with a hospital with high compliance and thousands of surgical patients. This is not necessarily the case, as rates based on small numbers of cases have a very wide margin of error and provide a less reliable snapshot of a hospital's performance. Further, research generally finds that outcomes, such as death following surgery, improve with volume of procedures performed. This report only addresses the risk of infection and not general outcomes at a particular hospital. If you are choosing where to have surgery, you may wish to look at additional performance measures for the hospitals available to you. Good sources of information include:

• Consumer Reports Health
• Agency for Healthcare Research and Quality Compendium
• Hospital Compare

Discrepancies in the total number of hospitals included in the state and national sections of this report

The total number of hospitals included in the report varies due to decisions we made to provide a complete picture of every U.S. hospital's report on these measures. The "national" page, which summarizes and compares information for each state, includes calculated averages that do not include hospitals that submitted no data (indicated with an "NA" on the state-specific pages). In other words, the averages are calculated based only on hospitals that submitted data. However, on the detail pages of each state, we chose to list these "NA" hospitals because we wanted consumers to be able to find their hospital's results, even if that hospital submitted no data. Thus the total number of hospitals within each state will not be the same on the national and state pages.

In our report's state by state analysis of the high and low compliance hospitals and the variations among hospitals within a state, we omitted hospitals reporting for less than 25 patients.

How we calculated compliance averages

State Patient Average

(Displayed on the National Detail page, Patient Average tab) For a particular performance measure, reporting period, and state, an estimate of the percentage of all eligible patients in that state who received the right treatment procedure. Computed as the weighted average of the compliance rates for all reporting hospitals within the state, where each hospital is weighted by the number of surgical patients sampled from that hospital. In that way, a hospital's influence on the State Patient Average is proportional to the number of surgeries reported for that hospital.

State Hospital Average

(Displayed on the National Detail page, Hospital Average tab) For a particular performance measure, reporting period, and state, the average compliance rate of reporting hospitals in that state. Computed as the sum of compliance rates for all reporting hospitals in the state divided by the number of reporting hospitals in the state.

National Patient Average

(Displayed on the National Detail page, Hospital Average tab, at the bottom) For a particular performance measure and reporting period, the average compliance rate of all reporting hospitals in the country. Computed as the sum 6 of the compliance rates of all reporting hospitals in the country divided by the number of reporting hospitals nationally.

National Hospital Average

(Displayed on the National Detail page, Patient Average tab) For a particular performance measure and reporting period, an estimate of the percentage of all eligible patients nationally who received the right treatment procedure. Computed as the weighted average of the compliance rates for all reporting hospitals nationally, where each hospital is weighted by the number of surgical patients sampled from that hospital. In that way, a hospital's influence on the National Patient Average is proportional to the number of surgeries reported for that hospital.

Dates associated with the data

This report allows consumers to view progress achieved by state and by individual hospital since CMS began collecting this data. The dates reflect the month and year that the data was submitted and not the dates coinciding with the data. Each quarter, the data is updated to provide a rolling 12-month compliance score. That is, when data for a new quarter is posted, it is averaged with data from the prior three quarters, so that each reporting interval in this report reflects 12 months of performance.

Below are the exact dates that each reporting period in the report represents:

Technical expert

Dale Bratzler, DO, MPH, Chief Executive Officer of the Oklahoma Foundation for Medical Quality and prior medical director for the organization's state and national quality improvement efforts. Dr. Bratzler provided technical support for this report. Dr. Bratzler provides clinical and technical support for local and national quality improvement initiatives including the National Surgical Care Improvement Project. He was the lead author of "Use of Antimicrobial Prophylaxis for Major Surgery," Arch Surg/Vol, 140, Feb 2005, which established the baseline for the SCIP project.

ACKNOWLEDGEMENTS

This report and website are the result of collaboration among Consumers Union (CU) staff across departments. The website was created by Greg Foster, Web Developer. The following Safe Patient Project staff contributed to the report and website refinements: Lisa McGiffert, Michael McCauley, Suzanne Henry, and Daniela Nuñez; Consumer Reports Health Rating Center staff Abbe Herzig, John Santa, and Doris Peter. We would like to thank Mike Saccucci, Keith Newsom-Stewart and Dylan Chang for their consultation and input on analyzing the data. We also would like to thank Dr. Dale Bratzler for his expert advice on this report and CMS staff Neil Gittings and Benedicta Abel-Steinberg, who provided information about the data used in the report and website.

1. Conversation with Neil S. Gittings, CMS, [9-10-08]
2. Bratzler, Dale W, et al, "Use of Antimicrobial Prophylaxis for Major Surgery," Arch Sug/Vol, 140, Feb 2005, p. 178.
3. Conversation with Dr. Dale Bratzler, 4-2-09 and 9-4-08.