An Electronic Health Record Tool Designed to Improve Pediatric Hospital Discharge has Low Predictive Utility for Readmissions

Journal of Hospital Medicine 13(11). 2018 November;:779-782. Published online first August 29, 2018 | 10.12788/jhm.3043

November 1, 2018|Journal of Hospital Medicine

Mark S. Brittan, MD, MPH;Sara Martin, RN, BSN, CPN;Leslie Anderson BSN, RN-C;Angela Moss, MS;Michelle R. Torok, PhD

We developed an electronic health record tool to improve pediatric hospital discharge. This tool flags children with three components that might complicate discharge: home health, polypharmacy (≥6 medications), or non-English speaking caregiver. The tool tallies components and displays them as a composite score of 0-3 points. We describe the tool’s development, implementation, and an evaluation of its predictive utility for 30-day unplanned readmissions in 29,542 discharged children. Of these children, 28% had a composite score of 1, 8% a score ≥2, and 4% were readmitted. The odds of readmission was significantly higher in children with composite score of 1 versus 0 (odds ratio [OR]: 1.7; 95% CI, 1.5-2) and ≥2 versus 0 (OR 4.2; 95% CI 3.6-4.9). The C-statistic for this model was 0.62. Despite the positive association of the score with readmission, the tool’s discriminatory performance is low. Additional research is needed to evaluate its practical benefit for improving the quality of hospital discharge.

RESULTS

Cohort Characteristics

Analysis was restricted to patients with at least 30 days of available follow-up time after the index admission (N = 29,542 patients; Figure). Patient characteristics from the index admission are shown in the Table. The median age was 5 years, and median LOS was 2 days. A total of 19% of patients were discharged with ≥6 discharge medications, 15% received ≥1 home-health orders, and 10% were documented to have a non-English speaking family caregiver. Almost 28% had a composite score of 1 and 8% a score ≥2. The unplanned 30-day readmission rate was 4%. In bivariate analysis, children with readmission had longer LOS, more discharge medications, and more home care than children without readmission. Caregiver language preference was not associated with readmission.

ROC analysis indicated that dichotomizing number of medications at ≥6 vs. <6 and home health at 0 vs. ≥1 categories maximized the sensitivity and specificity for predicting 30-day unplanned readmissions. In predictive logistic regression analysis, the odds of readmission was significantly higher in children with a composite score of 1 vs. 0 (odds ratio [OR], 1.7; 95% CI 1.5-2) and a score of ≥2 vs. 0 (OR, 4.2; 95% CI, 3.6-4.9). The c statistic for this model was 0.62, and the Brier score was 0.037. Internal validation of the predictive logistic regression model yielded identical results.

DISCUSSION

We describe the development of an electronic decision support tool designed to facilitate hospital discharge. The tool alerts inpatient teams to the presence of potentially complex home cares and language barriers but has low discriminatory performance for unplanned readmissions.

In retrospect, the model’s weaker performance is not surprising given the lack of extensive and rigorous statistical analysis, which is customary for predictive model development.¹¹ This study demonstrates that predictors with ORs indicating a strong relationship (eg, OR > 3.0) may not successfully discriminate between those with and without an outcome of interest.¹² Using a composite score ≥1 to define “high risk” misses intervening on half of readmitted children and leads to intervention on almost 10,000 nonreadmitted children; a threshold of ≥2 misses 82% of readmitted children and leads to intervention on more than 2,000 nonreadmitted children. Published pediatric models with superior predictive performance (AUC, 0.71-0.78) would be more suitable for risk assessment.^13-15

Since implementation, we have not audited frequency of the tool’s use or whether care is changed with its use. Such feedback would be a first step in demonstrating its utility in QI. Our organization is undertaking an overhaul of the discharge process to reduce unnecessary discharge delays, with plans to actively incorporate the instrument in workflows. For example, the tool can be used to prompt more timely and complete translation and interpretation, verify home-care order accuracy, and flag appropriate families for early and optimal teaching in uses of home-care equipment. The medication component can be used to improve safe discharge practices for children with polypharmacy (eg, as a reminder to fill prescriptions and review the accuracy of medication lists prior to discharge). These interventions may generate more impact on discharge efficiency and family reported measures of satisfaction or discharge readiness than readmission outcomes.

Despite its potential benefit in QI, this instrument will need further validation to ensure that it helps target factors that it intends to capture. About 75% of patients were missing home-care values in the original data and were assumed to have not received home health. While there was no missing data for medication number or language, we opted not to assess accuracy of these variables. Therefore, patients may have been misclassified due to clinical documentation omissions or errors.

The instrument’s framework is relatively simple and should reduce barriers to implementation elsewhere. However, this tool was developed for one setting, and the design may require adjustment for other environments. Regional or institutional variation in home-health eligibility or clinical documentation may impact home-care and medication scores. The score may change at discharge if home-health or medication orders are modified late. The tool performs none of the following: measurement of regimen complexity, identification of high-risk medications, distinguishing of new versus preexisting medications/home care, nor measurement of health literacy, parent education, or psychosocial risk. Adding these features might enhance the model. Finally, readmission rates did not rise linearly with each added point. A more sophisticated scoring system (eg, differentially weighting each risk factor) may also improve the performance of the tool.

Despite these limitations, we have implemented a real-time electronic tool with practical potential to improve the discharge process but with low utility for distinguishing readmissions. Additional validation and research is needed to evaluate its impact on hospital discharge quality metrics and family reported outcome measures.

References

Download Article