What would other emergency stroke teams do? Using explainable machine learning to understand variation in thrombolysis practice

Abstract

Objectives To understand between-hospital variation in thrombolysis use among patients in England and Wales who arrive at hospital within 4 hours of stroke onset.

Design Machine learning was applied to the Sentinel Stroke National Audit Programme (SSNAP) data set, to learn which patients in each hospital would likely receive thrombolysis.

Setting All hospitals (n=132) providing emergency stroke care in England and Wales. Thrombolysis use in patients arriving within 4 hours of known or estimated stroke onset ranged from 7% to 49% between hospitals.

Participants 88,928 stroke patients recorded in the national stroke audit who arrived at hospital within 4 hours of stroke onset, from 2016 to 2018.

Intervention Extreme Gradient Boosting (XGBoost) machine learning models, coupled with a SHAP model for explainability.

Main Outcome Measures Shapley (SHAP) values, providing estimates of how patient features, and hospital identity, influence the odds of receiving thrombolysis.

Results The XGBoost/SHAP model revealed that the odds of receiving thrombolysis reduced 9 fold over the first 120 minutes of arrival-to-scan time, varied 30 fold depending on stroke severity, reduced 3 fold with estimated rather than precise stroke onset time, fell 6 fold with increasing pre-stroke disability, fell 4 fold with onset during sleep, fell 5 fold with use of anticoagulants, fell 2 fold between 80 and 110 years of age, reduced 3 fold between 120 and 240 minutes of onset-to-arrival time, and varied 13 fold between hospitals. The hospital attended explained 56% of the variance in between-hospital thrombolysis use, adding in other hospital processes explained 74%, the patient population alone explained 36%, and the combined information from both patient population and hospital processes explained 95% of the variance in between-hospital thrombolysis use. Patient SHAP values expose how suitable a patient is considered for thrombolysis. Hospital SHAP values expose the threshold at which patients are likely to receive thrombolysis.

Conclusions Using explainable machine learning, we have identified that the majority of the between-hospital variation in thrombolysis use in England and Wales, for patients arriving with time to thrombolyse, may be explained by differences in in-hospital processes and differences in attitudes to judging suitability for thrombolysis.