Measuring Hospital Quality: Can Medicare Data Substitute for All‐Payer Data?

Monitoring quality of care across institutions and over time and examining the correlates of quality are critical to pursuing effective policy to improve quality (Institute of Medicine Committee on Quality of Health Care in America 2001; Kohn et al. 2000). Medical record abstraction has been the “gold standard” for constructing quality measures but the costs associated with abstracting data from the patient's chart makes its use infeasible in monitoring quality and overall health system performance and examining the factors that influence quality. Researchers wishing to conduct analysis on large samples of hospitals have turned to less expensive and more readily available administrative data, primarily patient discharge abstracts, to construct measures of hospital quality (Agency for Healthcare Research and Quality 2000; Ball et al. 1998; Geraci 2000; Iezzoni, Daley, Heeren, Foley, Hughes et al. 1994; Iezzoni, Daley, Heeren, Foley, Fisher et al. 1994; Johantgen et al. 1998; Kuykendall et al. 1995; Silber and Rosenbaum 1997). Over 40 states now collect discharge abstracts on all hospitalized patients in acute care hospitals. These data vary in completeness, number of primary and secondary diagnoses and procedures reported, presence of other patient information, such as race/ethnicity and insurer, years available, and cost. The Centers for Medicare and Medicaid Services' (CMS) MedPAR system contains information on hospital discharges for all Medicare patients. These data are relatively inexpensive, consistently coded, available for virtually all acute care hospitals in the United States, and have been used in many studies (Lawthers et al. 2000; Romano et al. 1994; Romano et al. 1995; Weingart et al. 2000). There are, however, important differences between the state and national discharge data on Medicare patients. For example, the public-use MedPAR data do not include information on dates for procedures and have fewer coded secondary diagnoses and procedures than most state datasets. Nevertheless, the quality of patient care based on Medicare data is often regarded as a surrogate measure of the quality of care for all hospitalized patients. Although the data on Medicare patients contained within all-patient state datasets are generally consistent with information on Medicare patients in the CMS Medicare data (Medstat Group Research and Policy Division 2000), and there is some evidence that hospital admission patterns for all patients can be predicted from the admission patterns of Medicare patients (Radany and Luft 1993), it is an empirical question whether Medicare data can be used as a close substitute for all-patient data for hospital quality studies. Quality measures have been used in studies to assess quality in specific hospitals and in studies of hospital characteristics associated with quality care. We focus on the second type of study and assess whether all-patient and Medicare data provide the same results in regression-based studies of correlates of quality across a range of measures. We analyze data from a sample of all-patient discharge abstracts for hospitals in 11 states, and patients in a national sample drawn from MedPAR data, examining three samples of patients: the 11-state all-patient sample, the Medicare patients in the 11-state data (11-state Medicare sample), and national MedPAR sample. Our quality indicators were developed and tested in a larger study that examined the association of patient outcomes and nurse staffing in acute care hospitals (Needleman et al. 2002; Needleman et al. 2001). Our analytic strategy is first to compare rates of adverse outcomes and results from regression analysis of outcomes on nurse staffing in the 11-state all-patient sample to those from analysis of the 11-state Medicare sample. This comparison allows us to draw conclusions on how closely Medicare patients are a surrogate for all patients in the same sample of hospitals using consistently coded discharge data and identical measures of nurse staffing. Because researchers working with Medicare data will likely use data from the CMS national MedPAR files, and because less information is available on these abstracts than in most state discharge abstracts, we compare adverse outcome rates and regression results in the national MedPAR sample with those from our 11-state Medicare and 11-state all-patient samples to determine if the results from the MedPAR and state data are consistent. We find some differences in regression results between the MedPAR and two 11-state samples and conduct additional analyses to determine the source of these differences.