Nephrotoxin-Related Acute Kidney Injury and Predicting High-Risk Medication Combinations in the Hospitalized Child
BACKGROUND: In the hospitalized patient, nephrotoxin exposure is one potentially modifiable risk factor for acute kidney injury (AKI). Clinical decision support based on nephrotoxin ordering was developed at our hospital to assist inpatient providers with the prevention or mitigation of nephrotoxin-related AKI. The initial decision support algorithm (Algorithm 1) was modified in order to align with a national AKI collaborative (Algorithm 2).
OBJECTIVE: Our first aim was to determine the impact of this alignment on the sensitivity and specificity of our nephrotoxin-related AKI detection system. Second, if the system efficacy was found to be suboptimal, we then sought to develop an improved model.
DESIGN: A retrospective cohort study in hospitalized patients between December 1, 2013 and November 30, 2015 (N = 14,779) was conducted.
INTERVENTIONS: With the goal of increasing nephrotoxin-related AKI detection sensitivity, a novel model based on the identification of combinations of high-risk medications was developed.
RESULTS: Application of the algorithms to our nephrotoxin use and AKI data resulted in sensitivities of 46.9% (Algorithm 1) and 43.3% (Algorithm 2, P = .22) and specificities of 73.6% and 89.3%, respectively (P < .001). Our novel AKI detection model was able to deliver a sensitivity of 74% and a specificity of 70%.
CONCLUSIONS: Modifications to our AKI detection system by adopting Algorithm 2, which included an expanded list of nephrotoxins and equally weighting each medication, did not improve our nephrotoxin-related AKI detection. It did improve our system’s specificity. Sensitivity increased by >50% when we applied a novel algorithm based on observed data with identification of key medication combinations.
© 2019 Society of Hospital Medicine
Study Population
This was a retrospective cohort study examining all patients ages 0-21 years admitted to SCH between December 1, 2013 and November 30, 2015. The detection system was modified to align with the national pediatric AKI collaborative, Nephrotoxic Injury Negated by Just-in-Time Action (NINJA) in November 2014. Both acute care and intensive care patients were included (data not separated by location). Patients who had end-stage kidney disease and were receiving dialysis and patients who were evaluated in the emergency department without being admitted or admitted as observation status were excluded from analysis. Patients were also excluded if they did not have a baseline serum creatinine as defined below.
Study Measures
AKI is defined at SCH using the Kidney Disease: Improving Global Outcomes Stage 1 criteria as a guideline. The diagnosis of AKI is based on an increase in the baseline serum creatinine by 0.3 mg/dL or an increase in the serum creatinine by >1.5 times the baseline assuming the incoming creatinine is 0.5 mg/dL or higher. For our definition, the increase in serum creatinine needs to have occurred within a one-week timeframe and urine output is not a diagnostic criterion.15 Baseline serum creatinine is defined as the lowest serum creatinine in the previous six months. Forty medications were classified as nephrotoxins based on previous analysis16 and adapted for our institutional formulary.
Statistical Analysis
To evaluate the efficacy of our systems in detecting nephrotoxin-related AKI, the sensitivity and the specificity using both our original algorithm (Algorithm 1) and the modified algorithm (Algorithm 2) were generated on our complete data set. To test sensitivity, the proportion of AKI patients who would trigger alert using Algorithm 1 and then with Algorithm 2 was identified. Similarly, to test specificity, the proportion of non-AKI patients who did not trigger an alert by the surveillance systems was identified. The differences in sensitivity and specificity between the two algorithms were evaluated using two-sample tests of proportion.
The statistical method of Combinatorial Inference has been utilized in studies of cancer biology17 and in genomics.18 A variation of this approach was used in this study to identify the specific medication combinations most associated with AKI. First, all of the nephrotoxic medications and medication combinations that were prescribed during our study period were identified from a data set (ie, a training set) containing 75% of all encounters selected at random without replacement. Using this training set, the prevalence of each medication combination and the rate of AKI associated with each combination were identified. The predicted overall AKI risk of an individual medication is the average of all the AKI rates associated with each combination containing that specific medication. Also incorporated into the determination of the predicted AKI risk was the prevalence of that medication combination.
To test our model’s predictive capability, the algorithm was applied to the remaining 25% of the total patient data (ie, the test set). The predicted AKI risk was compared with the actual AKI rate in the test data set. Our model’s predictive capability was represented in a receiver operator characteristic (ROC) analysis. The goal was to achieve an area under the ROC curve (AUC) approaching one as this would reflect 100% sensitivity and 100% specificity, whereas an AUC of 0.5 would represent a random guess (50% chance of being correct).
Lastly, our final step was to use our model’s ROC curve to determine an optimal threshold of AKI risk for which to trigger an alert. This predicted risk threshold was based on our goal to increase our surveillance system’s sensitivity balanced with maintaining an acceptable specificity.
An a priori threshold of P = .05 was used to determine statistical significance of all results. Analyses were conducted in Stata 12.1 (StataCorp LP, College Station, Texas) and R 3.3.2 (R Foundation for Statistical Computing, Vienna, Austria). A sample data set containing replication code for our model can be found in an online repository (https://dataverse.harvard.edu/dataverse/chuan). This study was approved by the Seattle Children’s Institutional Review Board.
