Accelerating Improvement and Narrowing Gaps: Trends in Patients’ Experiences with Hospital Care Reflected in HCAHPS Public Reporting Marc N. Elliott, Christopher W. Cohea, William G. Lehrman, Elizabeth H. Goldstein, Paul D. Cleary, Laura A. Giordano, Megan K. Beckett, and AlanM. Zaslavsky
Objective. Measure HCAHPS improvement in hospitals participating in the second and fifth years of HCAHPS public reporting; determine whether change is greater for some hospital types. Data. Surveys from 4,822,960 adult inpatients discharged July 2007–June 2008 or July 2010–June 2011 from 3,541 U.S. hospitals. Study Design. Linear mixed-effect regression models with fixed effects for time, patient mix, and hospital characteristics (bedsize, ownership, Census division, teaching status, Critical Access status); random effects for hospitals and hospital-time interac- tions; fixed-effect interactions of hospital characteristics and patient characteristics (gender, health, education) with time predicted HCAHPS measures correcting for regression-to-the-mean biases. Data Collection Methods. National probability sample of adult inpatients in any of four approved survey modes. Principal Findings. HCAHPS scores increased by 2.8 percentage points from 2008 to 2011 in the most positive response category. Among the middle 95 percent of hospi- tals, changes ranged from a 5.1 percent decrease to a 10.2 percent gain overall. The greatest improvement was in for-profit and larger (200 or more beds) hospitals. Conclusions. Five years after HCAHPS public reporting began, meaningful improvement of patients’ hospital care experiences continues, especially among ini- tially low-scoring hospitals, reducing some gaps among hospitals. Key Words. HCAHPS, for-profit, improvement, bedsize
In 1995, the agency now called the Agency for Healthcare Research andQual- ity initiated the Consumer Assessment of Healthcare Providers and Systems (CAHPS) project to develop surveys and survey protocols for collecting
©Health Research and Educational Trust DOI: 10.1111/1475-6773.12305 RESEARCHARTICLE
Health Services Research
reliable and valid assessments of health care and services from consumers. As part of that project, the CAHPS Consortium and the Centers for Medicare and Medicaid Services (CMS) developed the HCAHPS (Hospital Consumer Assessment of Healthcare Providers and Systems) Survey to assess patients’ hospital experiences. CMS began national administration of the HCAHPS survey using a standardized protocol in 2006. Beginning in 2008, HCAHPS results have been publicly reported quarterly on the U.S. Department of Health and Human Services’ Hospital Compare Web site. Participation in public reporting increased from 2,521 hospitals inMarch 2008 to 3,851 in Jan- uary 2012 (Centers for Medicare and Medicaid Services 2012a, c). This large increase corresponds to the linkage of HCAHPS participation to hospitals’ annual payment update (APU). Acute care hospitals reimbursed under the Inpatient Prospective Payment System were required to collect HCAHPS data beginning in July 2007 to receive their full APU; starting in March 2009, they were required to publicly report HCAHPS results (Giordano et al. 2010). These data permit a national assessment of patients’ experience of hos- pital care and enable assessments of changes in care quality (Giordano et al. 2010).
Public reporting of performance information is intended to inform consumer decisions and to help providers and payers monitor and improve the quality of care (McGlynn 2003). Public reporting of perfor- mance data allows patients and their intermediaries, such as employers, to compare and choose higher performing providers. Public performance data also allow hospitals and providers to identify and focus on areas that require improvement (Marshall et al. 2003). Both of these processes have the potential to improve quality-of-care experiences (Berwick, James, and Coye 2003). Other changes to the hospital landscape attributable to the Affordable Care Act are the consolidation of hospital systems, growth of contracted groups of physicians, and the decline of private practice physi- cians. Some expect that these changes will lead to more incentives for
Address correspondence to Marc N. Elliott, Ph.D., RAND Corporation, 1776 Main St., PO Box 2138, Santa Monica, CA 90407-2138; e-mail: [email protected] Christopher W. Cohea, M.S., and Laura A. Giordano, R.N., M.B.A., are with the Surveys, Research & Analysis Division, Health Services Advisory Group, Phoenix, AZ. William G. Lehrman, Ph.D., and Elizabeth H. Goldstein, Ph.D., are with the Center for Beneficiary Choices Department, Centers for Medicare and Medi- caid Services, Baltimore,MD. Paul D. Cleary, Ph.D., is with the School of Public Health, Yale Uni- versity, New Haven, CT. Megan K. Beckett, Ph.D., is also with RAND Corporation, Santa Monica, CA. Alan M. Zaslavsky, Ph.D., is with the Harvard Medical School, Department of Health Care Policy, Boston,MA.
Accelerating Improvement and Narrowing Gaps 1851
hospitals and medical staffs to meet quality, efficiency, and patient experi- ence goals (American Hospital Association 2013).
Prior analyses of HCAHPS data identify patient- and hospital-level characteristics that are associated with patient care experiences. Patient characteristics associated with patient experiences include age, education, and service line (O’Malley et al. 2005; Elliott et al. 2009). Nonprofit hospi- tals, hospitals with fewer Medicaid patients, and smaller hospitals have bet- ter patient experiences ( Jha et al. 2008). These same hospital-level characteristics may also be the most relevant to improvement in HCAHPS scores. In particular, we hypothesize that for-profit hospitals are likely to be especially attuned to financial incentives and, therefore, most likely to improve their scores once such incentives are enacted. Because large hospi- tals have the infrastructure for quality improvement initiatives and are more likely to have more resources to focus on factors affecting revenues, they may be more likely to improve than smaller hospitals. Hospitals that serve a large proportion of minority patients may face greater political pressures from local, state, and national constituencies to provide better patient expe- riences, and, in some instances, high proportions of patients for whom Eng- lish is not the preferred language may influence quality improvement efforts. Public and nonprofit hospitals and those with large proportions of minority patients may face greater regulatory pressure to provide better patient experiences for minority patients in particular (e.g., through greater cultural competency) than private, for-profit hospitals and hospitals with lar- gely white patient populations.
Because of the growing interest in patient experiences in part due to reimbursements being linked to patient experience scores, there have been increased efforts to improve patient-centered care, for example, by supporting nurses during nurse shift-time transitions and standardizing patient education materials (Frampton et al. 2008; Ketelsen et al. 2014; see psi-network.org for additional examples). It is important to know whether such efforts have improved patient experiences nationally, and, if so, which hospitals and patient groups have improved the most.
Using data from the 2,774 hospitals that participated in public HCAHPS reporting in March 2008 and March 2009, Elliott et al. (2010) found modest improvements over that period for all HCAHPS measures except doctor communication. Although scores differed by hospital size and other hospital characteristics, improvements were fairly uniform across hospitals.
In this article, we describe the experiences of over 4 million patients dis- charged from hospitals that participated in the second and fifth years of
1852 HSR: Health Services Research 50:6 (December 2015)
HCAHPS public reporting and assess the changes in HCAHPS scores across different types of hospital and patient groups.
We analyzedHCAHPS survey data collected during two intervals: (1) patients discharged from July 2007 to June 2008 (hereafter “2008”) and (2) patients dis- charged from July 2010 to June 2011 (hereafter “2011”) from the 3,541 acute care hospitals that participated in HCAHPS public reporting in both 2008 and 2011. The first interval is the first complete year of data after HCAHPS participation stabilized at its current levels. The HCAHPS Survey was admin- istered to a random sample of eligible patients between 2 and 42 days after discharge. Eligible patients were at least 18 years old at the time of inpatient admission, had at least one overnight stay with a nonpsychiatric principal diagnosis, and were discharged alive. For details of HCAHPS survey adminis- tration at the time, see HCAHPS Quality Assurance Guidelines V7.0 (Centers for Medicare and Medicaid Services 2012b); V9.0 applies currently. Nation- ally, there were 2,129,296/2,693,664 survey responses in 2008/2011, with overall response rate (RR) of 32 percent each year. The 2008HCAHPS scores were publicly reported in March 2009 and represent the second year of pub- licly reported data for most hospitals and the first year for about one-fourth of hospitals. The 2011 scores were publicly reported inMay 2012.
The primary outcome analyzed is a summary measure that averages eight HCAHPS components: the overall rating of the hospital; the six com- posite measures (“Communication with doctors,” “Communication with nurses,” “Responsiveness of hospital staff,” “Pain management,” “Communi- cation about medicines,” and “Discharge information”); and an average of the two stand-alone measures (“Cleanliness” and “Quietness” of the hospital envi- ronment). Survey response options are Never, Sometimes, Usually, and Always for all composite/report items other than the discharge items, for which the options are Yes or No. For the global rating, response options are numbers from 0 through 10. The average is then scaled to a potential range of 0 to 100. HCAHPS items and details of survey implementation protocols can be found at www.hcahpsonline.org.
Using two mixed-effect linear regression models, we analyze changes in hospital scores from 2008 to 2011. Because each hospital selects one of four approved survey modes (mail, telephone, mail with telephone follow-up, or active interactive voice response) for all of its patients, HCAHPS items are
Accelerating Improvement and Narrowing Gaps 1853
adjusted for survey mode prior to analysis (Elliott et al. 2009). Model 1 uses the standard HCAHPS patient-mix adjustors (service line, age group, educa- tion, self-reported health status, language spoken at home, response percentile (rank-ordered time between patient discharge and survey completion), and age by service line interactions) as fixed effects to control for differences in response tendencies associated with patient characteristics not under the con- trol of the hospital. It also includes an indicator for 2011 to measure change from 2008 and a hospital random effect to remove confounding between patient-mix adjustors and hospital quality.
We estimate 2008 and 2011-minus-2008 variance components using an unstructured covariance matrix, which allows us to estimate the disattenuated correlations in hospital performance across years. These correlations correct for the underestimation that would otherwise occur with hierarchical data (Woodhouse et al. 1996). We also use these results to compute an approximate range of change for the middle 95 percent of hospitals. The mixed-effect model with random slopes for change-over-time provides disattenuated esti- mates of change that remove any bias in those estimates of change that would be associated with sampling error (or regression to the mean).
Model 2 assesses whether the patient and hospital characteristics are associated with differential improvement, 2008–2011. Model 2, the full model with unstructured covariance, adds to Model 1 hospital characteristics (bed- size [100–199, 200+ vs. <100], a for-profit ownership status indicator, a teach- ing hospital indicator, the proportion of patients that are surgical, the proportion of patients that are maternity, a Critical Access Hospital indicator, Census Division [eight indicators relative to Pacific], and the proportions of patients who are Asian/Pacific Islander [API], black, and Hispanic [relative to non-Hispanic white plus others]), additional patient characteristics (female gender and race/ethnicity indicators), the interactions of hospital characteris- tics with time (2011 vs. 2008), and selected interactions of patient characteris- tics with time (gender, race/ethnicity, education and self-rated health, each parameterized linearly for power).
Hospital bedsize is parameterized (<100, 100–199, 200+) based on previ- ous HCAHPS findings (Lehrman et al. 2010). Critical Access Hospitals, typi- cally rural, are certified by CMS to receive cost-based reimbursement to ensure Medicare beneficiaries’ access to health care services. Patient-level interactions with time include the twomost important case mix adjustors (edu- cation, health status; Elliott et al. 2009) and interactions related to monitoring trends in HCAHPS health disparities (gender and race/ethnicity; Goldstein et al. 2010; Elliott et al. 2012). We include indicators and time interactions for
1854 HSR: Health Services Research 50:6 (December 2015)
Census divisions to control for large-scale secular trends in improvement that may vary by region.
Interactions of hospital characteristics with year estimate overall improvement at the hospital level; interactions of patient-level characteristics with year estimate differential improvement within hospitals. These interac- tions allow us to test for differing amounts of improvement. We illustrate trends in improvement by hospital characteristics by plotting mean patient- mix adjusted scores by selected hospital characteristics and year.
In the study years, approximately 37 percent of the hospitals have 200+ beds and about 19 percent are for-profit (Table 1). Among HCAHPS respondents, 37 percent are male, 78 percent are non-Hispanic white (hereafter “white”), 8 percent Hispanic, and 8 percent non-Hispanic black (hereafter “black”). Med- ian age is between 55 and 64 years. Just over half of respondents had attended at least some college, and about 40 percent rated their health as “excellent” or “very good.”
Unadjusted means for the patient experience summary measure are 84.2 percent in 2008 and 85.7 percent in 2011, with an improvement of 1.49 percentage points over 3 years (data not shown).
Table 2 presents the estimates from Model 1 of the relationships between the patient experience summary measure and survey year and patient-mix variables. The estimated adjusted improvement between 2008 and 2011 is 1.65 (p < .0001). This change is equivalent to an improvement of 0.498 of the hospital-level standard deviation over 3 years. Put differently, for a hospital at the 50th percentile in ranking in 2008, the average 3-year improvement would have increased their rank by 19 percentile points (to the 69th percentile) if other hospitals had not changed (a median 2011 score would have been 69th percentile in 2008). In general, at the median, a 1 per- centage point increase on the hospital-level patient experience summary scale translates into about a 12 percentile point improvement in rank, with improve- ments in rank somewhat smaller for hospitals further from the median.
Public reports of patient experiences highlight the percentage of patients (the “top-box” score) who selected the most positive response, such as “Always,” rather than the means that we used to maximize the power of these analyses. Analyses not presented here show that a 1 percent increase in the points earned under linear scoring translates into approximately a
Accelerating Improvement and Narrowing Gaps 1855
Table 1: Characteristics of HCAHPS Hospitals and Survey Participants
Pooled HCAHPS, 2008 and 2011 Unweighted %
Hospitals (n = 3,541) Bed size
<200 beds 63 200+ beds 37
Profit status Nonprofit/public 81 For-profit 19
Teaching hospital 29 Critical Access Hospital 12 Census division
New England 5 Mid-Atlantic 11 South Atlantic 16 East North Central 16 East South Central 9 West North Central 10 West South Central 14 Mountain 7 Pacific 11
Patients (n = 4,822,960) Gender
Male 37 Female 63
Race/Ethnicity Hispanic 8 Black 8 Asian/Pacific Islander 2 White 78 Other 3
Age 18–44 23 45–54 12 55–64 17 65–74 21 75–84 19 85 or older 8
Education <8th grade 6 Some high school 11 High school grad 33 Less than BA 27 BA 12 >4 years college 12
1856 HSR: Health Services Research 50:6 (December 2015)
Table 1: Continued
Pooled HCAHPS, 2008 and 2011 Unweighted %
Overall health Excellent 14 Very good 27 Good 31 Fair 21 Poor 7
Language other than Englishmainly spoken at home 5 Hospital service
Maternity 12 Surgical 34 Medical 54
Table 2: Model 1: Regression Coefficients (Standard Errors) for Mixed- Effect Linear Regression Model of Patient Experience Summary Measure: 2011 Indicator with Patient-Mix Adjustors and Hospital Random Effects (n = 4,822,960)
Coefficient (Standard Error)
Year (2008 reference) 2011 1.65 (0.01)***
Patient-level measures Age category (85+ reference)
18–24 !3.17 (0.05)*** 25–34 !1.57 (0.04)*** 35–44 !0.75 (0.04)*** 45–54 0.21 (0.03)*** 55–64 1.53 (0.03)*** 65–74 1.87 (0.03)*** 75–84 1.14 (0.03)***
General health status (5 = excellent, 1 = poor) !2.62 (0.01)*** Education (1 = noHS degree, 6 = >4 year college) !1.14 (0.00)*** Service line (medical reference)
Maternity 6.89 (0.07)*** Surgical 2.17 (0.05)***
Response percentile !4.99 (0.06)*** Non-English primary language 1.20 (0.03)*** Age by service line interactions
Maternity 9 age (linear categories) !0.07 (0.01)*** Surgical 9 age (linear categories) !0.92 (0.03)***
Hospital variance component (random effect) 11.32 (0.28)***
***p < .001.
Accelerating Improvement and Narrowing Gaps 1857
1.7 percentage point increase in top box scores (such as “Always” responses), so the mean 1.65 percent increase in points earned (linearly scored) is equiva- lent to about a 1.65 9 1.7 = 2.81 percentage point increase in top box scores.
Additional results (not shown) indicate that the estimated 2008–2011 change, adjusting for sampling error, for the middle 95 percent of hospitals ranges from a !3.0 percent decrease to a +6.0 percent increase in the percent- age of possible points. Thus, the average hospital increased the rate of “Always” responses by 2.8 percentage points, with a range of !5.1 to +10.2 percent for the middle 95 percent of hospitals.
Table 3 shows Model 2, which examines interactions of patient and hos- pital characteristics with time. Similar to previous analyses of HCAHPS data (Lehrman et al. 2010), scores are lower for larger hospitals (p < .0001 for each of 100–199 and 200+ beds). There is regional variation, with scores highest in states in the East South Central (p < .0001) and West South Central (p < .0001) divisions and lowest in the Mid-Atlantic, Mountain, and Pacific divisions. Also, as seen in earlier data (Elliott et al. 2010), while overall HCAHPS scores are lower in hospitals that treat more minority rather than white/other patients, within a given hospital, patient experiences are best for black patients (p < .0001), nearly as good for Hispanic patients (p < .0001), and slightly lower for API patients (p = .01), all compared with white/other patients.
The amount of improvement varies substantially across types of hospi- tals. The factors most strongly associated with improvement are bedsize and ownership. Hospitals with 200+ beds improved by an average of 1.2 points more than smaller hospitals over 3 years (p < .0001), such that 38 percent of the 2008 advantage associated with small hospitals (controlling for other hos- pital-level factors) disappeared by 2011. Medium-sized hospitals improved by an intermediate amount. For-profit hospitals improved by 1.3 points more than nonprofit hospitals (private or government) over 3 years (p < .0001), so that 81 percent of the 2008 advantage associated with nonprofit status disap- peared over 3 years.
Of the three Census divisions with the lowest 2008 scores, two (Mid- Atlantic and Pacific) improved an average amount, and the third (Mountain) improved the most of all divisions. The two highest scoring divisions in 2008 (East South Central and West South Central) improved the least 2008–2011. Critical Access Hospitals, which had above-average 2008 scores, improved more than other hospitals (+0.5 points, p = .0032). Hospitals that treated many API patients, which had below-average 2008 scores, improved much more than hospitals that did not (+0.6 points per 10 percent API patients,
1858 HSR: Health Services Research 50:6 (December 2015)
Table 3: Model 2: Regression Coefficients (Standard Errors) for Mixed- Effect Linear Regression Model of Patient Experience Summary Measure: Interactions of Hospital and Patient Characteristics with 2011 Indicator (n = 4,822,960)
Coefficient (Standard Error)
Year (2008 reference) 2011 !0.85 (0.22)***
Patient-level measures Age category (85+ reference)
18–24 !3.49 (0.05)*** 25–34 !1.88 (0.04)*** 35–44 !1.17 (0.04)*** 45–54 !0.17 (0.03)*** 55–64 1.20 (0.03)*** 65–74 1.61 (0.03)*** 75–84 1.00 (0.03)***
General health status (5 = Excellent, 1 = Poor) !2.69 (0.01)*** Education (1 = NoHS degree, 6 = >4 year college) !1.14 (0.01)*** Service line (Medical reference)
Maternity 7.00 (0.07)*** Surgical 2.44 (0.05)***
Response percentile !5.47 (0.06)*** Non-English primary language 0.72 (0.04)*** Age by service line interactions
Maternity 9 age (linear categories) !0.69 (0.03)*** Surgical 9 age (linear categories) !0.11 (0.01)***
Female !1.36 (0.02)*** Race/ethnicity (non-Hispanic white reference)
Asian/Pacific Islander !0.19 (0.07)* Black 2.33 (0.04)*** Hispanic 1.79 (0.05)***
Hospital-level characteristics Bedsize (1–99 Beds reference)
100–199 !2.21 (0.14)*** 200+ !3.09 (0.15)***
For-profit !1.65 (0.14)*** Teaching hospital !0.21 (0.13) Census divisions (Pacific reference)
New England 1.25 (0.31)*** Mid-Atlantic !0.41 (0.25) South Atlantic 0.47 (0.24)* East North Central 0.76 (0.23)** East South Central 2.69 (0.27)*** West North Central 1.49 (0.26)*** West South Central 2.72 (0.23)*** Mountain !0.27 (0.27)
Accelerating Improvement and Narrowing Gaps 1859
p < .0001). On the other hand, hospitals that served a large proportion of black patients improved somewhat less than hospitals that did not (!0.1 points per 10 percent black patients, p = .0019).
Within hospitals, differential improvements by patient type, while statis- tically significant, are generally smaller in magnitude than hospital-level
Table 3: Continued
Coefficient (Standard Error)
Proportion patients-surgical 1.86 (0.31)*** Proportion patients-maternity 0.64 (0.47) Critical Access Hospital 1.47 (0.20)*** Proportion patients-Asian Pacific Islander !10.40 (1.14)*** Proportion patients-black !5.53 (0.47)*** Proportion patients-Hispanic !6.74 (0.46)***
Interactions between patient-level variables and time Female 9 2011 !0.02 (0.03) Education (linear) 9 2011 0.10 (0.01)*** Health (linear) 9 2011 0.16 (0.01)*** Asian Pacific Islander 9 2011 !0.33 (0.10)*** Black 9 2011 !0.08 (0.05) Hispanic 9 2011 !0.15 (0.06)**
Interactions between hospital level characteristics and time Bedsize [relative to 1–99 beds]
100–199 9 2011 0.68 (0.12)*** 200+ 9 2011 1.16 (0.12)***
For-profit status 9 2011 1.33 (0.12)*** Teaching hospital 9 2011 0.21 (0.11)* Census divisions [relative to Pacific]
New England 9 2011 !0.40 (0.24) Mid-Atlantic 9 2011 !0.13 (0.19) South Atlantic 9 2011 0.35 (0.18) East North Central 9 2011 0.23 (0.18) East South Central 9 2011 !0.53 (0.21)* West North Central 9 2011 !0.06 (0.20) West South Central 9 2011 !0.71 (0.18)*** Mountain 9 2011 0.68 (0.21)**
Proportion patients-surgical 9 2011 0.85 (0.28)** Proportion patients-maternity 9 2011 1.25 (0.42)** Critical Access Hospital 9 2011 0.47 (0.16)** Proportion patients-Asian Pacific Islander 9 2011 5.62 (0.91)*** Proportion patients-black 9 2011 !1.17 (0.38)** Proportion patients-Hispanic 9 2011 0.68 (0.37) 2008Hospital variance component 9.18 (0.23)*** 2011–2008Hospital variance component (random effect) 4.85*** 2008 and 2011–2008 covariance (random effect) !4.55***
*p < .05, **p < .01, ***p < .001.
1860 HSR: Health Services Research 50:6 (December 2015)
changes. Improvements in patient experiences were somewhat greater for less healthy patients than healthier patients (p < .0001). Care experiences for patients who reported poor health increased an average of 0.6 points more over 3 years than care for patients in excellent health, closing 6 percent of that gap over 3 years.
Figures 1 and 2 illustrate change from 2008 to 2011 in adjusted HCAHPS summary scores based on bedsize and profit status. Unlike the results ofModel 2 in Table 3, these trends do not adjust for other hospital-level characteristics but consider only one hospital characteristic at a time. Figure 1 shows that a 3.1 percent mean difference between hospitals with <200 beds and those with 200+ beds in 2008 narrowed to a 2.1 percent difference by 2011 due to greater improvement in larger hospitals. Figure 2 illustrates con- vergence between nonprofit and for-profit hospitals, such that the advantage of nonprofits over for-profit hospitals in 2008 almost disappeared by 2011 due to small mean improvement among nonprofits in the face of larger improve- ment by for-profit hospitals. The vertical marks on each figure correspond to 95 percent confidence intervals; in no instances do these confidence intervals overlap for a given year.
Covariance estimates for Model 2 show that 2011 scores are correlated 0.69 with 2008 scores after correcting for regression to the mean. Changes from
Figure 1: Mean Adjusted HCAHPS Score, by Bedsize by Year
Note. Vertical lines indicate 95% confidence intervals for each of the four point estimates.
Accelerating Improvement and Narrowing Gaps 1861
2008 to 2011 are correlated 0.68 with 2008 scores, reflecting greater improve- ment among hospitals with lower initial scores, also after controlling for regres- sion to the mean. This pattern results in less variation at the hospital level in 2011 than in 2008, with the true (disattenuated) hospital-level standard devia- tions in performance falling 27 percent from 3.03 in 2008 to 2.22 in 2011.
Previous research in a smaller sample of 2,774 hospitals found modest but sig- nificant improvements in patient experience of care during the first 2 years of public reporting of HCAHPS scores (Elliott et al. 2010). Elliott et al. (2010) predicted more improvement after public reporting (2008) because hospitals would have more time to implement quality improvement efforts in anticipa- tion of the inclusion of HCAHPS measures in the Hospital Value-Based Purchasing (HVBP) program enacted by the Patient Protection and Afford- able Care Act of 2010 (P.L. 111-148; Section 3001). In this study of 3,541 hospitals, we find continued and greater improvement from 2008 to 2011. In a CMS-sponsored demonstration project in 2003, the effect of financial
Figure 2: Mean Adjusted HCAHPS Score, by Profit Status by Year
Note. Vertical lines indicate 95% confidence intervals for each of the four point estimates. Neither the two 2011 confidence intervals nor the two 2008 confidence intervals overlap.
1862 HSR: Health Services Research 50:6 (December 2015)
incentives and public reporting on quality improvement was greater than the effect of public reporting alone (Lindenauer et al. 2007). With HCAHPS scores directly affecting hospital payment beginning in October 2012, it is likely that improvement will continue.
Not all hospitals improved, and some improved more than others. The most important predictors of improvement were organizational- or hospital- level, rather than patient-level, factors. As we hypothesized, for-profit and lar- ger hospitals, each of which had worse patient experiences in 2007 ( Jha et al. 2008; Lehrman et al. 2010) and 2008 (in this study), improvedmore than non- profit and smaller hospitals in a context where the percentage of hospitals with these characteristics did not change much. For example, smaller hospitals (with <200 beds) made up 63 percent of participating hospitals in both 2008 and 2011. Larger, for-profit hospitals may have had more resources to imple- ment quality improvement efforts than other hospitals. There also was a ten- dency for initially lower-scoring Census divisions (especially Mountain) to improve more than initially high-scoring divisions (East South Central and West South Central), which improved the least 2008–2011. Regional differ- ences may reflect variation in patient-centered care or in the organization and financing of health care.
Analyses of change by race/ethnicity partially supported our hypothe- ses: hospitals that served higher proportions of API and Hispanic patients (associated with below average performance in 2008) improved more than other hospitals, but the opposite was true for hospitals serving more black patients (associated with average performance in 2008). It may be that much of the improvement in hospitals serving large proportions of Latinos and Asians was related to linguistic or cultural competence interventions that ben- efit primarily non-English speakers and which had no counterpart in hospitals serving primarily black patients. It is also possible that due to greater residen- tial and service segregation of blacks than other groups (Elliott et al. 2008), together with lower income and more poorly reimbursed health insurance, such hospitals have fewer resources to devote to quality improvement.
Differences in amount of improvement for different patient groups within hospitals were generally small, especially compared to hospital-level differences in improvement associated with hospital-level characteristics. Improvement was slightly greater for patients in poorer health and with higher education. The summary measure, which is similar to what is used for HCAHPS value-based purchasing, was selected to maximize power to detect and summarize statistically significant interactions with time. Nevertheless, future analyses should test for differential improvement by individual
Accelerating Improvement and Narrowing Gaps 1863
measures to better understand whether improvement is heterogeneous across measures by hospital or patient characteristics.
Improvement was greater for hospital types that initially scored lower on HCAHPS, even when using methods that adjust for regression-to- the-mean bias; this is reflected in scores that are considerably more similar across hospitals in 2011 than in 2008. Our results are similar to prior studies that documented greater improvement on clinical measures among initially lower-performing health plans and hospitals ( Jencks et al. 2000; Williams et al. 2005). Although prior studies could not rule out the possibility that this greater improvement in clinical process measures may have been due to more thorough documentation by hospitals, this cannot explain improvements in patient-reported experiences. Thus, improvement was generally greater where it was most needed. Future quality improvement efforts could examine the approaches of hospitals that made the largest improvements.
A limitation of our study is that response rates are low, though similar to those from other patient experience surveys ( Jha et al. 2008; Roland et al. 2009). Furthermore, response rates are only weakly associated with nonre- sponse bias in similar probability sample surveys adhering to high process standards of survey methodology (Groves and Peytcheva 2008; Groves et al. 2009; Johnson and Wislar 2012; Davern 2013; Halbesleben and Whitman 2013). Any remaining nonresponse bias would tend to be similar within the same hospital over time so that substantial nonresponse bias in estimated changes in scores over time is especially unlikely.
Under Hospital Value-Based Purchasing (HVBP), HCAHPS perfor- mance will account for 30 percent of participating hospitals’ payment incen- tive in FY 2013, 2014, and 2015 (Centers for Medicare and Medicaid Services 2011). HVBP explicitly rewards both performance level and improvement in performance. The HVBP scoring system, coupled with improving HCAHPS performance, implies that to maintain their relative position hospitals will have to continue to improve patient experience of care. The public reporting of HCAHPS data, which began in March 2008, as well as the anticipation of Hospital Value-Based Purchasing, may have motivated hospital quality improvement efforts, most notably among hospitals with lower initial scores. Five years after the public reporting of HCAHPS results began, the HCAHPS program shows increasing promise of tangibly improving patients’ experience of hospital care and reducing variability in quality of patient-centered care among hospitals.
1864 HSR: Health Services Research 50:6 (December 2015)
Joint Acknowledgment/Disclosure Statement: This study was funded by CMS con- tract HHSM-500-2011-AZ10C to the Health Services Advisory Group. Eliza- beth Goldstein and William Lehrman are employees of the sponsoring agency, the CMS. The authors have no conflicts of interest to report.We thank Fergal McCarthy,M.Phil., for assistance with manuscript preparation.
Disclosures: None. Disclaimers: None.
American Hospital Association. 2013. Hospitals: The Changing Landscape is Good for Patients & Health Care. Washington, DC: American Hospital Association.
Berwick, D. M., B. James, and M. J. Coye. 2003. “Connections between Quality Mea- surement and Improvement.”Medical Care 41 (1 Suppl): I30–8.
Centers for Medicare and Medicaid Services. 2011. Open Door Forum: Hospital Value- Based Purchasing. Baltimore, MD: Centers forMedicare andMedicaid Services.
Centers for Medicare and Medicaid Services. 2012a. “HCAHPS Fact Sheet” [accessed on May 29, 2012]. Available at http://www.hcahps.org/files/HCAHPS%20Fact %20Sheet%20May%202012.pdf
Centers for Medicare andMedicaid Services. 2012b. “HCAHPS QAGuidelines Page” [accessed onMay 29, 2012]. Available at http://www.hcahpsonline.org/qaguide- lines.aspx
Centers for Medicare andMedicaid Services. 2012c. “HCAHPSWebsite” [accessed on May 29, 2012]. Available at http://www.hcahps.org/home.aspx
Davern, M. 2013. “Nonresponse Rates Are a Problematic Indicator of Nonresponse Bias in Survey Research.”Health Services Research 48 (3): 905–12.
Elliott, M. N., B. K. Finch, D. Klein, S. Ma, D. P. Do, M. K. Beckett, N. Orr, and N. Lurie. 2008. “Sample Designs for Measuring the Health of Small Racial/Ethnic Subgroups.” Statistics in Medicine 27 (20): 4016–29.
Elliott, M. N., A.M. Zaslavsky, E. Goldstein, W. Lehrman, K. Hambarsoomians, M. K. Beckett, and L. Giordano. 2009. “Effects of SurveyMode, Patient Mix, and Non- response on CAHPS Hospital Survey Scores.”Health Services Research 44 (2 Pt 1): 501–18.
Elliott, M. N., W. G. Lehrman, E. H. Goldstein, L. A. Giordano, M. K. Beckett, C. W. Cohea, and P. D. Cleary. 2010. “Hospital Survey Shows Improvements in Patient Experience.”Health Affairs 29 (11): 2061–7.
Elliott, M. N., W. G. Lehrman, M. K. Beckett, E. Goldstein, K. Hambarsoomian, and L. A. Giordano. 2012. “Gender Differences in Patients’ Perceptions of Inpatient Care.”Health Services Research 47 (4): 1482–501.
Accelerating Improvement and Narrowing Gaps 1865
Frampton, S., S. Guastello, C. Brady, M. Hale, S. Horowitz, S. B. Smith, and S. Stone. 2008. Patient-Centered Care: Improvement Guide. Derby, CT: Planetree Inc and Picker Institute.
Giordano, L. A., M. N. Elliott, E. Goldstein, W. G. Lehrman, and P. A. Spencer. 2010. “Development, Implementation, and Public Reporting of the HCAHPS Sur- vey.”Medical Care Research and Review: MCRR 67 (1): 27–37.
Goldstein, E., M. N. Elliott, W. G. Lehrman, K. Hambarsoomian, and L. A. Giordano. 2010. “Racial/Ethnic Differences in Patients’ Perceptions of Inpatient Care Using the HCAHPS Survey.” Medical Care Research and Review: MCRR 67 (1): 74–92.
Groves, R. M., and E. Peytcheva. 2008. “The Impact of Nonresponse Rates on Nonre- sponse Bias: AMeta-Analysis.” Public Opinion Quarterly 72 (2): 167–89.
Groves, R. M., F. Fowler Jr., M. Couper, J. Lepkowski, E. Singer, and R. Tourangeau. 2009. Survey Methodology. 2d ed. New York: JohnWiley and Sons.
Halbesleben, J. R., and M. V. Whitman. 2013. “Evaluating Survey Quality in Health Services Research: A Decision Framework for Assessing Nonresponse Bias.” Health Services Research 48 (3): 913–30.
Jencks, S. F., T. Cuerdon, D. R. Burwen, B. Fleming, P. M. Houck, A. E. Kuss- maul, D. S. Nilasena, D. L. Ordin, and D. R. Arday. 2000. “Quality of Medical Care Delivered to Medicare Beneficiaries: A Profile at State and National Levels.” Journal of the American Medical Association 284 (13): 1670–6.
Jha, A. K., E. J. Orav, J. Zheng, and A. M. Epstein. 2008. “Patients’ Perception of Hospital Care in the United States.” New England Journal of Medicine 359 (18): 1921–31.
Johnson, T. P., and J. S. Wislar. 2012. “Response Rates and Nonresponse Errors in Surveys.” Journal of the American Medical Association 307 (17): 1805–6.
Ketelsen, L., K. Cook, and B. Kennedy. 2014. The HCAHPS Handbook, Tactics to Improve Quality and the Patient Experience. 2d ed. Pensacola, FL: Fire Starter Publishing.
Lehrman, W. G., M. N. Elliott, E. Goldstein, M. K. Beckett, D. J. Klein, and L. A. Giordano. 2010. “Characteristics of Hospitals Demonstrating Superior Perfor- mance in Patient Experience and Clinical Process Measures of Care.” Medical Care Research and Review: MCRR 67 (1): 38–55.
Lindenauer, P. K., D. Remus, S. Roman, M. B. Rothberg, E. M. Benjamin, A. Ma, and D. W. Bratzler. 2007. “Public Reporting and Pay for Performance in Hospital Quality Improvement.” New England Journal of Medicine 356 (5): 486–96.
Marshall, M. N., P. G. Shekelle, H. T. Davies, and P. C. Smith. 2003. “Public Reporting on Quality in the United States and the United Kingdom.” Health Affairs (Mill- wood) 22 (3): 134–48.
McGlynn, E. A. 2003. “Introduction and Overview of the Conceptual Framework for a National Quality Measurement and Reporting System.”Medical Care 41 (1 Sup- pl): I1–7.
1866 HSR: Health Services Research 50:6 (December 2015)
O’Malley, A. J., A. M. Zaslavsky, M. N. Elliott, L. Zaborski, and P. D. Cleary. 2005. “Case-Mix Adjustment of CAHPS Hospital Survey.” Health Services Research 40 (6p2): 2162–81.
Roland, M., M. Elliott, G. Lyratzopoulos, J. Barbiere, R. A. Parker, P. Smith, P. Bower, and J. Campbell. 2009. “Reliability of Patient Responses in Pay for Performance Schemes: Analysis of National General Practitioner Patient Survey Data in Eng- land.” BMJ 339: b3851.
Williams, S. C., S. P. Schmaltz, D. J. Morton, R. G. Koss, and J. M. Loeb. 2005. “Quality of Care in U.S. Hospitals as Reflected by Standardized Measures, 2002-2004.”New England Journal of Medicine 353 (3): 255–64.
Woodhouse, G., M. Yang, H. Goldstein, and J. Rasbash. 1996. “Adjusting for Measure- ment Error in Multilevel Analysis.” Journal of the Royal Statistical Society Series A-Statistics in Society 159: 201–12.
Additional supporting information may be found in the online version of this article:
Appendix SA1: AuthorMatrix.
Accelerating Improvement and Narrowing Gaps 1867
Copyright of Health Services Research is the property of Wiley-Blackwell and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder’s express written permission. However, users may print, download, or email articles for individual use.
Are you busy and do not have time to handle your assignment? Are you scared that your paper will not make the grade? Do you have responsibilities that may hinder you from turning in your assignment on time? Are you tired and can barely handle your assignment? Are your grades inconsistent?
Whichever your reason may is, it is valid! You can get professional academic help from our service at affordable rates. We have a team of professional academic writers who can handle all your assignments.
Our essay writers are graduates with diplomas, bachelor, masters, Ph.D., and doctorate degrees in various subjects. The minimum requirement to be an essay writer with our essay writing service is to have a college diploma. When assigning your order, we match the paper subject with the area of specialization of the writer.
Delivering a high-quality product at a reasonable price is not enough anymore.
That’s why we have developed 5 beneficial guarantees that will make your experience with our service enjoyable, easy, and safe.
You have to be 100% sure of the quality of your product to give a money-back guarantee. This describes us perfectly. Make sure that this guarantee is totally transparent.Read more
Each paper is composed from scratch, according to your instructions. It is then checked by our plagiarism-detection software. There is no gap where plagiarism could squeeze in.Read more
Thanks to our free revisions, there is no way for you to be unsatisfied. We will work on your paper until you are completely happy with the result.Read more
Your email is safe, as we store it according to international data protection rules. Your bank details are secure, as we use only reliable payment systems.Read more
By sending us your money, you buy the service we provide. Check out our terms and conditions if you prefer business talks to be laid out in official language.Read more