Enhancing Early Sepsis Prediction with Temporal bias EHRs Data
DOI:
https://doi.org/10.71321/4t9gsy05Keywords:
Sepsis prediction, Transformer, Electronic health records, Deep learning, Interpretable modelAbstract
Background: Sepsis prediction models using electronic health records (EHRs) are often challenged by temporal biases from irregular data entry and severe class imbalance. This study develops a novel deep learning (DL) framework to address these specific challenges for accurate and early sepsis detection.
Methods: We propose a Feature-Wise Multi-Head Self-Attention Transformer (FW-MHSA-former) with an Adaptive Balance-Preserving Ensemble (ABPE). FW-MHSA-former mitigates temporal bias by applying self-attention across medical features to model correlations directly. ABPE resolves class imbalance by partitioning the majority class to train multiple models on balanced datasets, aggregating predictions via weighted voting. The framework was retrospectively validated on the MIMIC-IV dataset; Kaplan-Meier analysis assessed survival outcomes. Visualizing the attention-derived feature correlation matrix enhances interpretability.
Results: The proposed framework achieved a peak Area Under the Receiver Operating Characteristic Curve (AUROC) of 0.94. At this optimal performance, the model demonstrated a recall of 0.90 and an accuracy of 0.87. We compared our method with three classical models and three advanced attention/Transformer-based models. The proposed approach yielded consistently superior performance across all evaluation metrics, including accuracy and F1 score. The KM analysis confirmed that the model effectively stratified patients into high- and low-risk cohorts with statistically significant differences in survival outcomes (p < 0.001).
Conclusions: The proposed framework effectively and robustly predicts early sepsis. By addressing timestamp irregularities and class imbalance, it achieves superior accuracy and provides an interpretable tool to enhance clinical decision support in critical care.
References
[1] Duan, Y., Huo, J., Chen, M., Hou, F., Yan, G., Li, S., et al. (2023). Early prediction of sepsis using double fusion of deep features and handcrafted features. Applied intelligence (Dordrecht, Netherlands), 1–17. Advance online publication. https://doi.org/10.1007/s10489-022-04425-z
[2] Fleuren, L. M., Klausch, T. L. T., Zwager, C. L., Schoonmade, L. J., Guo, T., Roggeveen, L. F., et al. (2020). Machine learning for the prediction of sepsis: a systematic review and meta-analysis of diagnostic test accuracy. Intensive care medicine, 46(3), 383–400. https://doi.org/10.1007/s00134-019-05872-y
[3] Reyna, M. A., Josef, C. S., Jeter, R., Shashikumar, S. P., Westover, M. B., Nemati, S., et al. (2020). Early Prediction of Sepsis From Clinical Data: The PhysioNet/Computing in Cardiology Challenge 2019. Critical care medicine, 48(2), 210–217. https://doi.org/10.1097/CCM.0000000000004145
[4] Lauritsen, S. M., Kalør, M. E., Kongsgaard, E. L., Lauritsen, K. M., Jørgensen, M. J., Lange, J.,et al. (2020). Early detection of sepsis utilizing deep learning on electronic health record event sequences. Artificial intelligence in medicine, 104, 101820. https://doi.org/10.1016/j.artmed.2020.101820
[5] Wang, Y., Lu, X., Tan, J., He, J., Yin, H., Chen, P., et al. (2025). The cGAS-STING Pathway: Insights into Regulatory Mechanisms, Disease Dysregulation, and Therapeutic Development. Life Conflux, 2(1), e300. https://doi.org/10.71321/dr57c347
[6] Bomrah, S., Uddin, M., Upadhyay, U., Komorowski, M., Priya, J., Dhar, E., et al. (2024). A scoping review of machine learning for sepsis prediction- feature engineering strategies and model performance: a step towards explainability. Critical care (London, England), 28(1), 180. https://doi.org/10.1186/s13054-024-04948-6
[7] Deng, H. F., Sun, M. W., Wang, Y., Zeng, J., Yuan, T., Li, T., et al. (2021). Evaluating machine learning models for sepsis prediction: A systematic review of methodologies. iScience, 25(1), 103651. https://doi.org/10.1016/j.isci.2021.103651
[8] Delahanty, R. J., Alvarez, J., Flynn, L. M., Sherwin, R. L., & Jones, S. S. (2019). Development and Evaluation of a Machine Learning Model for the Early Identification of Patients at Risk for Sepsis. Annals of emergency medicine, 73(4), 334–344. https://doi.org/10.1016/j.annemergmed.2018.11.036
[9] Yan, M. Y., Gustad, L. T., & Nytrø, Ø. (2022). Sepsis prediction, early detection, and identification using clinical text for machine learning: a systematic review. Journal of the American Medical Informatics Association : JAMIA, 29(3), 559–575. https://doi.org/10.1093/jamia/ocab236
[10] Nemati, S., Holder, A., Razmi, F., Stanley, M. D., Clifford, G. D., & Buchman, T. G. (2018). An Interpretable Machine Learning Model for Accurate Prediction of Sepsis in the ICU. Critical care medicine, 46(4), 547–553. https://doi.org/10.1097/CCM.0000000000002936
[11] Shashikumar, S. P., Josef, C. S., Sharma, A., & Nemati, S. (2021). DeepAISE - An interpretable and recurrent neural survival model for early prediction of sepsis. Artificial intelligence in medicine, 113, 102036. https://doi.org/10.1016/j.artmed.2021.102036
[12] Lauritsen, S. M., Kristensen, M., Olsen, M. V., Larsen, M. S., Lauritsen, K. M., Jørgensen, M. J., et al. (2020). Explainable artificial intelligence model to predict acute critical illness from electronic health records. Nature communications, 11(1), 3852. https://doi.org/10.1038/s41467-020-17431-x
[13] Mao, Q., Jay, M., Hoffman, J. L., Calvert, J., Barton, C., Shimabukuro, D., et al. (2018). Multicentre validation of a sepsis prediction algorithm using only vital sign data in the emergency department, general ward and ICU. BMJ open, 8(1), e017833. https://doi.org/10.1136/bmjopen-2017-017833
[14] Tang, X., Zheng, D., Kebede, G. S., Li, Z., Li, X., Lu, C., et al. (2023). An automatic segmentation framework of quasi-periodic time series through graph structure. Applied Intelligence, 53(20), 23482–23499. https://doi.org/10.1007/s10489-023-04814-y
[15] van de Sande, D., van Genderen, M. E., Huiskens, J., Gommers, D., & van Bommel, J. (2021). Moving from bytes to bedside: a systematic review on the use of artificial intelligence in the intensive care unit. Intensive care medicine, 47(7), 750–760. https://doi.org/10.1007/s00134-021-06446-7
[16] Lai, H., Wu, G., Zhong, Y., Chen, G., Zhang, W., Shi, S., et al. (2023). Red blood cell distribution width improves the prediction of 28-day mortality for patients with sepsis-induced acute kidney injury: A retrospective analysis from MIMIC-IV database using propensity score matching. Journal of intensive medicine, 3(3), 275–282. https://doi.org/10.1016/j.jointm.2023.02.005
[17] Oami, T., Imaeda, T., Nakada, T. A., Abe, T., Takahashi, N., Yamao, Y., et al. (2023). Mortality analysis among sepsis patients in and out of intensive care units using the Japanese nationwide medical claims database: a study by the Japan Sepsis Alliance study group. Journal of intensive care, 11(1), 2. https://doi.org/10.1186/s40560-023-00650-x
[18] Zohar, Y., Zilberman Itskovich, S., Koren, S., Zaidenstein, R., Marchaim, D., & Koren, R. (2021). The association of diabetes and hyperglycemia with sepsis outcomes: a population-based cohort analysis. Internal and emergency medicine, 16(3), 719–728. https://doi.org/10.1007/s11739-020-02507-9
[19] Das, P.P., Wiese, L., Mast, M., Böhnke, J., Wulff, A., Marschollek, M., et al. (2024). An attention-based bidirectional LSTM-CNN architecture for the early prediction of sepsis. International Journal of Data Science and Analytics, 20, 1841 - 1855. https://doi.org/10.1007/s41060-024-00568-z
[20] Zhao, Y., Wu, Y., Liu, M., Cai, X., Zhang, Y., Yuan, X. (2022). DEAR: Dual-Level Self-attention GRU for Online Early Prediction of Sepsis. In: Zhao, X., Yang, S., Wang, X., Li, J. (eds) Web Information Systems and Applications. WISA 2022. Lecture Notes in Computer Science, vol 13579. Springer, Cham. https://doi.org/10.1007/978-3-031-20309-1_37
[21] Rafiei, A., Rezaee, A., Hajati, F., Gheisari, S., & Golzan, M. (2021). SSP: Early prediction of sepsis using fully connected LSTM-CNN model. Computers in biology and medicine, 128, 104110. https://doi.org/10.1016/j.compbiomed.2020.104110
[22] Johnson, A. E. W., Bulgarelli, L., Shen, L., Gayles, A., Shammout, A., Horng, S., et al. (2023). MIMIC-IV, a freely accessible electronic health record dataset. Scientific data, 10(1), 1. https://doi.org/10.1038/s41597-022-01899-x
[23] Seymour, C. W., Liu, V. X., Iwashyna, T. J., Brunkhorst, F. M., Rea, T. D., Scherag, A., et al. (2016). Assessment of Clinical Criteria for Sepsis: For the Third International Consensus Definitions for Sepsis and Septic Shock (Sepsis-3). JAMA, 315(8), 762–774. https://doi.org/10.1001/jama.2016.0288
[24] Seymour, C. W., Kennedy, J. N., Wang, S., Chang, C. H., Elliott, C. F., Xu, Z., et al. (2019). Derivation, Validation, and Potential Treatment Implications of Novel Clinical Phenotypes for Sepsis. JAMA, 321(20), 2003–2017. https://doi.org/10.1001/jama.2019.5791
[25] Gyawali, B., Ramakrishna, K., & Dhamoon, A. S. (2019). Sepsis: The evolution in definition, pathophysiology, and management. SAGE open medicine, 7, 2050312119835043. https://doi.org/10.1177/2050312119835043
[26] Camacho-Cogollo, J. E., Bonet, I., Gil, B., & Iadanza, E. (2022). Machine Learning Models for Early Prediction of Sepsis on Large Healthcare Datasets. Electronics, 11(9), 1507. https://doi.org/10.3390/electronics11091507
[27] Singer, M., Deutschman, C. S., Seymour, C. W., Shankar-Hari, M., Annane, D., Bauer, M., et al. (2016). The Third International Consensus Definitions for Sepsis and Septic Shock (Sepsis-3). JAMA, 315(8), 801–810. https://doi.org/10.1001/jama.2016.0287
[28] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., et al. (2017). Attention is all you need. In Advances in Neural Information Processing Systems, 5998-6008. https://doi.org/10.48550/arXiv.1706.03762
[29] Mikolov, T., Chen, K., Corrado, G., & Dean, J. (2013). Efficient estimation of word representations in vector space. arXiv, 1301.3781. https://arxiv.org/abs/1301.3781
[30] Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., et al. (2020). Language models are few-shot learners. arXiv, 2005.14165. https://arxiv.org/abs/2005.14165
[31] Radford, A., Wu, J., Child, R., Luan, D., Amodei, D., & Sutskever, I. (2019). Language models are unsupervised multitask learners. OpenAI.
[32] Wang, X., Thakker, M., Chen, Z., Kanda, N., Eskimez, S.E., Chen, S., et al. (2023). SpeechX: Neural Codec Language Model as a Versatile Speech Transformer. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 32, 3355-3364. https://doi.org/10.1109/TASLP.2024.3419418
[33] Yang, D., Tian, J., Tan, X., Huang, R., Liu, S., Chang, X., et al. (2023). UniAudio: An Audio Foundation Model Toward Universal Audio Generation. ArXiv, 2310.00704. https://arxiv.org/abs/2310.00704
[34] Chen, C.-F., Fan, Q., & Panda, R. (2021). CrossViT: Cross-attention multi-scale vision transformer for image classification. arXiv, 2103.14899. https://arxiv.org/abs/2103.14899
[35] Xin, C., Liu, Z., Zhao, K., Miao, L., Ma, Y., Zhu, X., et al. (2022). An improved transformer network for skin cancer classification. Computers in biology and medicine, 149, 105939. https://doi.org/10.1016/j.compbiomed.2022.105939
[36] Dutta, P., Sathi, K. A., Hossain, M. A., & Dewan, M. A. A. (2023). Conv-ViT: A Convolution and Vision Transformer-Based Hybrid Feature Extraction Method for Retinal Disease Detection. Journal of imaging, 9(7), 140. https://doi.org/10.3390/jimaging9070140
[37] Wang, L., Fang, S., Meng, X., & Li, R. (2022). Building extraction with vision transformer. IEEE Transactions on Geoscience and Remote Sensing, 60, 1–11. https://doi.org/10.1109/TGRS.2022.3186634
[38] Bhopale, A. P., & Tiwari, A. (2024). Transformer based contextual text representation framework for intelligent information retrieval. Expert Systems with Applications, 238, 121629. https://doi.org/10.1016/j.eswa.2023.121629
[39] Kim, Y., Bang, S., Sohn, J., & Kim, H. (2022). Question answering method for infrastructure damage information retrieval from textual data using bidirectional encoder representations from transformers. Automation in Construction, 134, 104061. https://doi.org/10.1016/j.autcon.2021.104061
[40] Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., et al. (2021). An image is worth 16x16 words: Transformers for image recognition at scale. arXiv, 2010.11929. https://arxiv.org/abs/2010.11929
[41] Liu, Z., Lin, Y., Cao, Y., Hu, H., Wei, Y., Zhang, Z., et al. (2021). Swin transformer: Hierarchical vision transformer using shifted windows. 2021 IEEE/CVF International Conference on Computer Vision (ICCV), 9992–10002. https://doi.org/10.48550/arXiv.2103.14030
[42] Liu, Z., Hu, H., Lin, Y., Yao, Z., Xie, Z., Wei, Y., et al. (2022). Swin transformer V2: Scaling up capacity and resolution. 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 11999–12009. https://doi.org/10.1109/CVPR52688.2022.01170
[43] Nie, Y., Nguyen, N. H., Sinthong, P., & Kalagnanam, J. (2022). A time series is worth 64 words: Long-term forecasting with transformers. arXiv. https://doi.org/10.48550/arXiv.2211.14730
[44] Liu, Y., Hu, T., Zhang, H., Wu, H., Wang, S., Ma, L., et al. (2023). iTransformer: Inverted transformers are effective for time series forecasting. arXiv. https://doi.org/10.48550/arXiv.2310.06625
[45] Mo, Y., Wu, Q., Li, X., & Huang, B. (2021). Remaining useful life estimation via transformer encoder enhanced by a gated convolutional unit. J Intell Manuf 32, 1997–2006. https://doi.org/10.1007/s10845-021-01750-x
[46] Guo, M.-H., Xu, T.-X., Liu, J.-J., Liu, Z.-N., Jiang, P.-T., Mu, T.-J., et al. (2022). Attention mechanisms in computer vision: A survey. Computational Visual Media, 8(3), 331–368. https://doi.org/10.1007/s41095-022-0271-y
[47] Labach, A., Pokhrel, A., Huang, X., Zuberi, S., Yi, S., Volkovs, M., et al. (2023). DuETT: Dual Event Time Transformer for Electronic Health Records. arXiv, 2304.13017. https://doi.org/10.48550/arXiv.2304.13017
[48] Wang, Y., Huang, N., Li, T., Yan, Y., Zhang, X. (2024). Medformer: A Multi-Granularity Patching Transformer for Medical Time-Series Classification. arXiv, 2405.19363. https://doi.org/10.48550/arXiv.2405.19363
[49] Singer, M., Deutschman, C. S., Seymour, C. W., Shankar-Hari, M., Annane, D., Bauer, M., et al. (2016). The Third International Consensus Definitions for Sepsis and Septic Shock (Sepsis-3). JAMA, 315(8), 801–810. https://doi.org/10.1001/jama.2016.0287
Type
Published
Data Availability Statement
The MIMIC-IV dataset utilized in this study originates from the Beth Israel Deaconess Medical Center (BIDMC) and is managed by the Massachusetts Institute of Technology's Computer Science and Artificial Intelligence Laboratory (MIT CSAIL). Given that the dataset contains sensitive patient health information, researchers must apply for access and complete a Data Use Agreement (DUA) along with Collaborative Institutional Training Initiative (CITI) certification to ensure ethical usage and protection of privacy. Consequently, the raw data is not publicly available; however, authorized researchers may obtain access through the official MIMIC-IV portal at https://mimic.physionet.org/.
Issue
Section
License
Copyright (c) 2026 Life Conflux
This work is licensed under a Creative Commons Attribution 4.0 International License.
How to Cite
