An Electronic Health Record Tool Designed to Improve Pediatric Hospital Discharge has Low Predictive Utility for Readmissions

To generate the score, the clinical documentation of home-health orders categorizes children with home care. Each home-care equipment or service category is documented in separate flowsheet rows, allowing for identification of distinct categories (Table). Total parenteral nutrition, intravenous medications, and durable medical equipment and supplies are counted as home care. The number of discharge medications is approximated by inpatient medication orders and finalized as the number of discharge medication orders. The medications include new, historic, and as-needed medications (if included among discharge medication orders). Home oxygen is not counted as medication. The preferred language of the family caregiver is recorded during patient registration or by the admitting inpatient nurse and is gleaned from either of these flowsheet sources.

The electronic score is displayed within the EHR’s Discharge Readiness Report⁸ and updates automatically as relevant data are entered. The tool displays the individual components and as a composite of 0-3 points. To register a point in each category, a patient needs to exceed (1) the dichotomous discharge medications criterion (ie, ≥6 medications), (2) the dichotomous home-health order criterion (ie, ≥1 home-care order), and (3) to possess documentation of a non-English speaking caregiver. The tool serves as a visual reminder of discharge planning needs during daily coordination rounds attended by clinicians, nursing managers, case managers, and social workers. Case managers use the home-care alert to verify the accuracy of home-care orders.

Evaluation of Predictive Utility for Readmission

We performed a retrospective cohort study on patients aged 0-21 years who were discharged between January 1, 2014 and December 30, 2015. This study was performed to determine the optimal cut points for the continuous variables (discharge medications and home-care orders) and to evaluate the predictive value of the composite score.

Unplanned readmission within 30 days was used as the primary outcome. The index hospitalization was randomly selected for patients with >1 admissions to avoid biasing the results with multiple hospitalizations from individual patients.

Patient characteristics were summarized using percentages for categorical variables and the median and interquartile range (IQR) for continuous variables. We examined bivariate associations for each of the tool’s predictor elements with readmission using Chi-square and Wilcoxon tests (level of statistical significance was set at 0.05). Receiver operating characteristics (ROC) analyses established optimal dichotomization points for medications and home-care orders using the maximized Youden’s index (sensitivity + specificity – 1).⁹ Dichotomization of these variables was selected for ease of implementation and interpretation. Similarly, the composite score was treated as a categorical predictor and because sensitivity analysis using it as a continuous predictor did not improve the tool's discriminatory properties.

The area under the ROC curve (AUC) was estimated to evaluate the performance of the composite score (as a categorical variable) using a predictive logistic regression model. To establish internal validity, we performed 10-fold cross validation analysis.¹⁰ Measures of predictive performance included the c statistic and the Brier score. The c statistic represents the discriminative ability of a model. A model with AUC > 0.9 signifies high discrimination, 0.7-0.9 indicates moderate discrimination, and 0.5-0.7 suggests low discrimination. The Brier score is a measure of the overall accuracy of the predictions and ranges from 0 to 1, with 0 representing perfect accuracy.¹⁰ Analyses were performed using SAS v9.4.