Beyond Reporting Early Warning Score Sensitivity: The Temporal Relationship and Clinical Relevance of “True Positive” Alerts that Precede Critical Deterioration
BACKGROUND: Clinical deterioration is difficult to detect in hospitalized children. The pediatric Rothman Index (pRI) is an early warning score that incorporates vital signs, laboratory studies, and nursing assessments to generate deterioration alerts.
OBJECTIVES: (1) Evaluate the timing of pRI alerts and clinicians recognizing deterioration or escalating care prior to critical deterioration events (CDEs) and (2) determine whether the parameters triggering alerts were clinically related to deterioration.
DESIGN: CDEs are unplanned transfers to the intensive care unit with noninvasive ventilation, tracheal intubation, and/or vasopressor infusion in the 12 hours after transfer. Using one year of data from a large freestanding children’s hospital without the pRI, we analyzed CDEs that would have been preceded by pRI alerts. We (1) compared the timing of pRI alerts to time-stamped notes describing changes in patient status and orders reflecting escalations of care and (2) identified score component(s) that caused alerts to trigger and determined whether these were clinically related to CDE etiology.
RESULTS: Fifty CDEs would have triggered pRI alerts if the pRI had been in use (sensitivity 68%). In 90% of CDEs, the first clinician note reflecting change in patient status and/or the first order reflecting escalation of care preceded the first pRI alert. All of the vital sign and laboratory components of the pRI and 51% of the nursing components were clinically related to the etiology of the CDE.
CONCLUSIONS: Evidence that clinicians were aware of deterioration preceded pRI alerts in most CDEs that generated alerts in the preceding 24 hours.
© 2018 Society of Hospital Medicine.
Patient Characteristics
The 50 CDEs labeled as true positives occurred in 46 unique patients. Table 1 displays the event characteristics.
Acuity Alerts
A total of 79 pRI alerts preceded the 50 CDEs. Of these acuity alerts, 44 (56%) were medium acuity alerts, 17 (22%) were high acuity alerts, and 18 (23%) were very high acuity alerts. Of the 50 CDEs that would have triggered pRI alerts, 33 (66%) would have triggered a single acuity alert and 17 (34%) would have triggered multiple acuity alerts.
Of the 50 CDEs, 39 (78%) had a preceding acuity alert within 24 hours prior to the CDE. In these cases, the alert preceded the CDE by a median of 3.1 hours (interquartile range of 0.7 to 10.3 hours).
We assessed the score components that caused each alert to trigger. All of the vital sign and laboratory components were assessed as clinically related to the CDE’s etiology. By contrast, about half of nursing assessment components were assessed as clinically related to the etiology of the CDE (Table 2). Abnormal cardiac, respiratory, and neurologic assessments were most frequently assessed as clinically relevant.
Escalation Orders
To determine whether the pRI alert would have preceded the earliest documented treatment efforts, we restricted evaluation to the 39 CDEs that had at least one alert in the 24-hour window prior to the CDE. When we reviewed escalation orders placed by clinicians, we found that in 26 cases (67%), the first clinician order reflecting escalation of care would have preceded the first pRI alert within the 24-hour period prior to the CDE. In 13 cases (33%), the first pRI alert would have preceded the first escalation order placed by the clinician. The first pRI alert and the first escalation order would have occurred within the same 1-hour period in 6 of these cases.
Provider Notes
Temporal Relationships
In Supplement 2, we present the proportion of CDEs in which the order or note preceded the pRI alert for each abnormal organ system.
The Figure shows the temporal relationships among escalation orders, clinician notes, and acuity alerts for the 39 CDEs with one or more alerts in the 24 hours leading up to the event. In 21 cases (54%), both an escalation order and a note preceded the first acuity alert. In 14 cases (36%), either an escalation order or a note preceded the first acuity alert. In four cases (10%), the alert preceded any documented evidence that clinicians had recognized deterioration or escalating care.
