Do HCAHPS doctor communication scores reflect the communication skills of the attending on record? A cautionary tale from a tertiary-care medical service

Study Design

The study utilized a cross sectional design; physicians who consented were observed on rounds during 3 separate encounters, and we compared hospitalists’ 4HCS scores to their HCAHPS scores to assess the correlation. The study was approved by the Institutional Review Board of the Cleveland Clinic.

Population

The study was conducted at the main campus of the Cleveland Clinic. All physicians specializing in hospital medicine who had received 10 or more completed HCAHPS survey responses while rounding on a medicine service in the past year were invited to participate in the study. Participation was voluntary; night hospitalists were excluded. A research nurse was trained in the Four Habits Model²⁸ and in the use of the 4HCS coding scheme by the principal investigator. The nurse observed each physician and ascertained the presence of communication behaviors using the 4HCS tool. Physicians were observed between August 2013 and August 2014. Multiple observations per physician could occur on the same day, but only 1 observation per patient was used for analysis. Observations consisted of a physician’s first encounter with a hospitalized patient, with the patient’s consent. Observations were conducted during encounters with English-speaking and cognitively intact patients only. Resident physicians were permitted to stay and conduct rounds per their normal routine. Patient information was not collected as part of the study.

Measures

HCAHPS. For each physician, we extracted all HCAHPS scores that were collected from our hospital’s Press Ganey database. The HCAHPS survey contains 22 core questions divided into 7 themes or domains, 1 of which is doctor communication. The survey uses frequency-based questions with possible answers fixed on a 4-point scale (4=always, 3=usually, 2=sometimes, 1=never). Our primary outcome was the doctor communication domain, which comprises 3 questions: 1) During this hospital stay, how often did the doctors treat you with respect? 2) During this hospital stay, how often did the doctors listen to you? and 3) During this hospital stay, how often did the doctors explain things in a language you can understand? Because CMS counts only the percentage of responses that are graded “always,” so-called “top box” scoring, we used the same measure.

The HCAHPS scores are always attributed to the physician at the time of discharge even if he may not have been responsible for the care of the patient during the entire hospital course. To mitigate contamination from patients seen by multiple providers, we cross-matched length of stay (LOS) data with billing data to determine the proportion of days a patient was seen by a single provider during the entire length of stay. We stratified patients seen by the attending providers to less than 50%, 50% to less than 100%, and at 100% of the LOS. However, we were unable to identify which patients were seen by other consultants or by residents due to limitations in data gathering and the nature of the database.

The Four Habits. The Four Habits are: invest in the beginning, elicit the patient’s perspective, demonstrate empathy, and invest in the end (Figure 1). Specific behaviors for Habits 1 to 4 are outlined in the Appendix, but we will briefly describe the themes as follows. Habit 1, invest in the beginning, describes the ability of the physician to set a welcoming environment for the patient, establish rapport, and collaborate on an agenda for the visit. Habit 2, elicit the patient’s perspective, describes the ability of the physician to explore the patients’ worries, ideas, expectations, and the impact of the illness on their lifestyle. Habit 3, demonstrate empathy, describes the physician’s openness to the patient’s emotions as well as the ability to explore, validate; express curiosity, and openly accept these feelings. Habit 4, invest in the end, is a measure of the physician’s ability to counsel patients in a language built around their original concerns or worries, as well as the ability to check the patients’ understanding of the plan.^2,29-30

4HCS. The 4HCS tool (Appendix) measures discreet behaviors and phrases based on each of the Four Habits (Figure 1). With a scoring range from a low of 4 to a high of 20, the rater at bedside assigns a range of points on a scale of 1 to 5 for each habit. It is an instrument based on a teaching model used widely throughout Kaiser Permanente to improve clinicians’ communication skills. The 4HCS was first tested for interrater reliability and validity against the Roter Interaction Analysis System using 100 videotaped primary care physician encounters.²⁹ It was further evaluated in a randomized control trial. Videotapes from 497 hospital encounters involving 71 doctors from a variety of clinical specialties were rated by 4 trained raters using the coding scheme. The total score Pearson’s R and intraclass correlation coefficient (ICC) exceeded 0.70 for all pairs of raters, and the interrater reliability was satisfactory for the 4HCS as applied to heterogeneous material.³⁰