Reports From the Field

Evaluation of an Enhanced Discharge Summary Template: Building a Better Handoff Document




A 30-minute educational session was developed and delivered by the authors. The objectives of the didactic portion were to describe how discharge summaries can impact patient care, understand how discharge summaries serve as a handoff document, list the components of an effective discharge summary, and describe strategies to avoid common errors in writing discharge summaries. The session included a review of pertinent literature [1,12,13,21], an outline of discharge summary best-practices [22,25], and an introduction to the new template. Trainers reviewed strategies for keeping the discharge summary concise, including using problem-based formatting, focusing on active hospital problems, and eliminating unnecessary or redundant information. Participants were encouraged to complete their discharge summaries and directly contact outpatient providers within 24 hours of discharge. Following the didactic session, participants critically reviewed an example discharge summary and discussed what was done well, what was done poorly, and what strategies they would have used to make it a more effective handoff document. Residents rotating on the inpatient internal medicine services received the education during their mandatory monthly orientation. Faculty physicians were provided the education at a required section meeting.

Quality Scoring of Discharge Summaries and Analysis

To evaluate the quality of discharge summaries, we developed a scoring instrument to measure inclusion of 24 key elements (Table 1). The scoring instrument (available from the authors) was pilot tested by 4 general internal medicine physicians on 5 sample discharge summaries. After independent scoring, this group met with members of the research team to provide feedback. Iterative revisions were made to the scoring instrument until scorers reached consensus in their understanding and application of the scoring instrument. Each discharge summary received a quality score from 0 to 24, based on the number of elements found to be present. Secondary quality metrics included a global quality rating using a 1 to 5 scale (described in Results); frequency of redundant documentation of consultants and procedures; frequency of documentation of non-active, chronic conditions; the length of the discharge summary (total word count); and time to completion.

We analyzed a sample of discharge summaries completed during the 3-month period prior to the intervention and the 3-month period following the intervention. A non-stratified random technique was employed by an independent party to generate discharge summary samples from the EHR. Living patients discharged from the internal medicine services after an inpatient admission of at least 48 hours were eligible for inclusion. Each discharge summary was scored by 2 general internal medicine physicians. Each scoring dyad comprised one of the authors paired with a volunteer non–research team member who scored discharge summaries independently. Discordant results were examined by the dyad and settled by consensus.

Physician Survey

We surveyed inpatient and outpatient physicians to determine their views about discharge summaries and their views about the template before and after the intervention. Respondents were asked to indicate to what degree they agreed with statements using a 5-point Likert scale. An email containing a consent cover letter and a link to an anonymous online survey was sent to residents rotating on internal medicine services during the study period and all hospital medicine faculty. Outpatient providers affiliated with the hospital system were sent the survey if they had received at least 5 discharge summaries from the internal medicine services over the preceding 6 months. Post-intervention surveys were timed to capture responses after an adequate exposure to the enhanced discharge summary template. Inpatient physicians were re-surveyed 3 months after introduction of the enhanced discharge summary and outpatient providers were re-surveyed after 1 year.

Statistical Analysis

We reviewed 10 pre-intervention discharge summaries to estimate baseline discharge summary quality scores. Anticipating a two-fold improvement following the intervention [24], we calculated a goal sample size of 108 discharge summaries (54 pre- and 54 post-intervention) assuming alpha of 0.05 and 80% power using a two-tailed chi-square test. Expecting that some discharge summaries may not meet our inclusion criteria, 114 summaries (57 pre- and 57 post-intervention) were included in the final sample. All analyses were performed on Stata v10.1 (StataCorp; College Station, TX).

For discharge summary quality scoring, inter-rater reliability was measured by calculating the kappa statistic and percent agreement for scoring elements. Chi-square analysis was used to compare individual scoring elements before and after the intervention when the sample size was 5 or greater. Fisher’s exact test was used when the sample size was less than 5. Counts, including number of inactive diagnoses, redundant consults, redundant procedures, and total words were compared using univariate Poisson regression. Wilcoxon rank sum analysis was utilized to compare pre-intervention to post-intervention composite scores and global scores. Patient and provider characteristics were compared using the t-test, chi-square test, Fisher’s exact test, or Wilcoxon rank sum, as appropriate.

For the surveys, pre-intervention and post-intervention matched pairs were compared. Likert score responses were analyzed using the Wilcoxon signed-rank test.


Discharge Summary Quality Scores

Characteristics of the pre- and post-intervention discharge summaries are displayed in Table 2. Both samples were similar with respect to patient demographics, length of stay, medical complexity, and provider characteristics. The mean composite discharge summary quality score improved from 13.4 at baseline to 19.1 in the post-intervention sample (P < 0.001) (Table 3). Ten of 24 quality elements exhibited significant improvement following the intervention, but 3 items were documented less often after the intervention (Table 3).

The global rating of discharge summary quality improved from 3.04 to 3.46 (P = 0.010) (Table 4). Documentation of superfluous and redundant information decreased in the 3 areas evaluated: number of non-active, chronic diagnoses (2.33 to 1.35, P < 0.001), redundant consults (1.4 to 0.09, P < 0.001), and redundant procedures (0.74 to 0.26, P < 0.001). Inter-rater reliability was generally high for individual items, although kappa score was not calculable in one case and scores of zero were obtained for 3 highly concordant items. Inter-rater reliability was moderate for global rating (kappa = 0.59). The overall length of discharge summaries decreased from 717 to 701 words (P = 0.002). There was no significant change in time to discharge summary completion following the intervention (10.9 hours pre-intervention vs. 14.5 post-intervention, P = 0.605) (Table 4).

Next Article: