BACKGROUND AND OBJECTIVE: Severe courses of COVID-19 disease can lead to long-term complications. The post-acute phase of COVID-19 refers to the persistent or new symptoms. This problem is becoming more relevant with the increasing number of patients who have contracted COVID-19 and the emergence of new virus variants. In this case, preventive treatment with corticosteroids can be applied. However, not everyone benefits from the treatment, moreover, it can have severe side effects. Currently, no study would analyze who benefits from the treatment. METHODS: This work introduces a novel approach to the recommendation of Corticosteroid (CS) treatment for patients in the post-acute phase. We have used a novel combination of clinical data, including blood tests, spirometry, and X-ray images from 273 patients. These are very challenging to collect, especially from patients in the post-acute phase of COVID-19. To our knowledge, no similar dataset exists in the literature. Moreover, we have proposed a unique methodology that combines machine learning and deep learning models based on Vision Transformer (ViT) and InceptionNet, preprocessing techniques, and pretraining strategies to deal with the specific characteristics of our data. RESULTS: The experiments have proved that combining clinical data with CXR images achieves 8% higher accuracy than independent analysis of CXR images. The proposed method reached 80.0% accuracy (78.7% balanced accuracy) and a ROC-AUC of 0.89. CONCLUSIONS: The introduced system for CS treatment prediction using our neural network and learning algorithm is unique in this field of research. Here, we have shown the efficiency of using mixed data and proved it on real-world data. The paper also introduces the factors that could be used to predict long-term complications. Additionally, this system was deployed to the hospital environment as a recommendation tool, which admits the clinical application of the proposed methodology.

Brno University of Technology FEEC Dept of Telecommunications Technicka 12 Brno 616 00 Czech Republic

Center for Digital Health Palacky Univesity Olomouc Faculty of Medicine and Dentistry Hnevotinska 976 3 Olomouc 779 00 Czech Republic

Electrical Engineering Unit Faculty of Information Technology and Communication Sciences Tampere University Tampere 33720 Finland

Zobrazit více v PubMed

Ma Q., Liu J., Liu Q., Kang L., Liu R., Jing W., et al. Global percentage of asymptomatic SARS-CoV-2 infections among the tested population and individuals with confirmed COVID-19 diagnosis. JAMA Netw Open. 2021;4:e2137257. PubMed PMC

Desai A.D., Lavelle M., Boursiquot B.C., Wan E.Y. Long-term complications of COVID-19. Am J Physiol, Cell Physiol. 2022;322:C1–C11. PubMed PMC

Zhang C., Wu Z., Li J.-W., Tan K., Yang W., Zhao H., et al. Discharge may not be the end of treatment: pay attention to pulmonary fibrosis caused by severe COVID-19. J Med Virol. 2021;93:1378–1386. PubMed

Myall K.J., Mukherjee B., Castanheira A.M., Lam J.L., Benedetti G., Mak S.M., et al. Persistent post–COVID-19 interstitial lung disease. An observational study of corticosteroid treatment. Ann Am Thorac Soc. 2021;18:799–806. PubMed PMC

Bieksiene K., Zaveckiene J., Malakauskas K., Vaguliene N., Zemaitis M., Miliauskas S. Post COVID-19 organizing pneumonia: the right time to interfere. Medicina. 2021;57:283. PubMed PMC

Sarfraz A., Sarfraz Z., Razzack A.A., Patel G., Sarfraz M. Venous thromboembolism, corticosteroids and COVID-19: a systematic review and meta-analysis. Clin Appl Thromb/Hemost. 2021;27 PubMed PMC

Volmer T., Effenberger T., Trautner C., Buhl R. Consequences of long-term oral corticosteroid therapy and its side-effects in severe asthma in adults: a focused review of the impact data in the literature. Eur Respir J. 2018;52 PubMed

Mongardon N., Piagnerelli M., Grimaldi D., Perrot B., Lascarrou J.-B. Impact of late administration of corticosteroids in COVID-19 ARDS. Intensive Care Med. 2021;47:110–112. PubMed PMC

Tlayjeh H., Mhish O.H., Enani M.A., Alruwaili A., Tleyjeh R., Thalib L., et al. Association of corticosteroids use and outcomes in COVID-19 patients: a systematic review and meta-analysis. J Infect Publ Health. 2020;13:1652–1663. PubMed PMC

Nabahati M., Ebrahimpour S., Khaleghnejad Tabari R., Mehraeen R. Post-COVID-19 pulmonary fibrosis and its predictive factors: a prospective study. Egypt J Radiol Nucl Med. 2021;52:1–7.

dataset C.S. 2023. https://drive.google.com/drive/u/2/folders/1F9cjEcptMpc8nbFuKR_SQ0O-sY4Op3L4

Dansana D., Kumar R., Bhattacharjee A., Hemanth D.J., Gupta D., Khanna A., et al. Early diagnosis of COVID-19-affected patients based on X-ray and computed tomography images using deep learning algorithm. Soft Comput. 2020:1–9. PubMed PMC

Mettler F.A., Jr., Huda W., Yoshizumi T.T., Mahesh M. Effective doses in radiology and diagnostic nuclear medicine: a catalog. Radiology. 2008;248:254–263. PubMed

Murphy K., Smits H., Knoops A.J., Korst M.B., Samson T., Scholten E.T., et al. COVID-19 on chest radiographs: a multireader evaluation of an artificial intelligence system. Radiology. 2020;296:E166–E172. PubMed PMC

He K., Zhang X., Ren S., Sun J. Proceedings of the IEEE conference on computer vision and pattern recognition. 2016. Deep residual learning for image recognition; pp. 770–778.

Huang G., Liu Z., Van Der Maaten L., Weinberger K.Q. Proceedings of the IEEE conference on computer vision and pattern recognition. 2017. Densely connected convolutional networks; pp. 4700–4708.

Szegedy C., Vanhoucke V., Ioffe S., Shlens J., Wojna Z. Proceedings of the IEEE conference on computer vision and pattern recognition. 2016. Rethinking the inception architecture for computer vision; pp. 2818–2826.

Sitaula C., Hossain M.B. Attention-based VGG-16 model for COVID-19 chest X-ray image classification. Appl Intell. 2021;51:2850–2863. PubMed PMC

Nishio M., Kobayashi D., Nishioka E., Matsuo H., Urase Y., Onoue K., et al. Deep learning model for the automatic classification of COVID-19 pneumonia, non-COVID-19 pneumonia, and the healthy: a multi-center retrospective study. Sci Rep. 2022;12:8214. PubMed PMC

Nishio M., Noguchi S., Matsuo H., Murakami T. Automatic classification between COVID-19 pneumonia, non-COVID-19 pneumonia, and the healthy on chest X-ray image: combination of data augmentation methods. Sci Rep. 2020;10 PubMed PMC

Wehbe R.M., Sheng J., Dutta S., Chai S., Dravid A., Barutcu S., et al. DeepCOVID-XR: an artificial intelligence algorithm to detect COVID-19 on chest radiographs trained and tested on a large U.S. clinical data set. Radiology. 2021;299:E167. PubMed PMC

Zhang R., Tie X., Qi Z., Bevins N.B., Zhang C., Griner D., et al. Diagnosis of coronavirus disease 2019 pneumonia by using chest radiography: value of artificial intelligence. Radiology. 2021;298:E88–E97. PubMed PMC

Jia G., Lam H.-K., Xu Y. Classification of COVID-19 chest X-Ray and CT images using a type of dynamic CNN modification method. Comput Biol Med. 2021;134 PubMed PMC

Khan E., Rehman M.Z.U., Ahmed F., Alfouzan F.A., Alzahrani N.M., Ahmad J. Chest X-ray classification for the detection of COVID-19 using deep learning techniques. Sensors. 2022;22:1211. PubMed PMC

Gour M., Jain S. Uncertainty-aware convolutional neural network for COVID-19 X-ray images classification. Comput Biol Med. 2022;140 PubMed PMC

Sharma A., Singh K., Koundal D. A novel fusion based convolutional neural network approach for classification of COVID-19 from chest X-ray images. Biomed Signal Process Control. 2022;77 PubMed PMC

Bargshady G., Zhou X., Barua P.D., Gururajan R., Li Y., Acharya U.R. Application of CycleGAN and transfer learning techniques for automated detection of COVID-19 using X-ray images. Pattern Recognit Lett. 2022;153:67–74. PubMed PMC

Hussain E., Hasan M., Rahman M.A., Lee I., Tamanna T., Parvez M.Z. CoroDet: a deep learning based classification for COVID-19 detection using chest X-ray images. Chaos Solitons Fractals. 2021;142 PubMed PMC

Shamout F.E., Shen Y., Wu N., Kaku A., Park J., Makino T., et al. An artificial intelligence system for predicting the deterioration of COVID-19 patients in the emergency department. npj Digit Med. 2021;4:1–11. PubMed PMC

Soda P., D'Amico N.C., Tessadori J., Valbusa G., Guarrasi V., Bortolotto C., et al. AIforCOVID: predicting the clinical outcomes in patients with COVID-19 applying AI to chest-X-rays. An Italian multicentre study. Med Image Anal. 2021;74 PubMed PMC

Ahsan M.M., Alam T.E., Trafalis T., Huebner P. Deep MLP-CNN model using mixed-data to distinguish between COVID-19 and non-COVID-19 patients. Symmetry. 2020;12:1526.

Khan I.U., Aslam N., Anwar T., Alsaif H.S., Chrouf S.M.B., Alzahrani N.A., et al. Using a deep learning model to explore the impact of clinical data on COVID-19 diagnosis using chest X-ray. Sensors. 2022;22:669. PubMed PMC

Jiao Z., Choi J.W., Halsey K., Tran T.M.L., Hsieh B., Wang D., et al. Prognostication of patients with covid-19 using artificial intelligence based on chest x-rays and clinical data: a retrospective study. Lancet Digit Health. 2021;3:e286–e294. PubMed PMC

Chieregato M., Frangiamore F., Morassi M., Baresi C., Nici S., Bassetti C., et al. A hybrid machine learning/deep learning COVID-19 severity predictive model from CT images and clinical data. Sci Rep. 2022;12:1–15. PubMed PMC

Myska V., Genzor S., Mezina A., Burget R., Mizera J., Stybnar M., et al. Artificial-intelligence-driven algorithms for predicting response to corticosteroid treatment in patients with post-acute COVID-19. Diagnostics. 2023;13:1755. PubMed PMC

Jin C., Yu H., Ke J., Ding P., Yi Y., Jiang X., et al. Predicting treatment response from longitudinal images using multi-task deep learning. Nat Commun. 2021;12:1–11. PubMed PMC

Lou B., Doken S., Zhuang T., Wingerter D., Gidwani M., Mistry N., et al. An image-based deep learning framework for individualising radiotherapy dose: a retrospective analysis of outcome prediction. Lancet Digit Health. 2019;1:e136–e147. PubMed PMC

Gosselt H.R., Verhoeven M.M., Bulatović-Ćalasan M., Welsing P.M., de Rotte M.C., Hazes J.M., et al. Complex machine-learning algorithms and multivariable logistic regression on par in the prediction of insufficient clinical response to methotrexate in rheumatoid arthritis. J Personalized Med. 2021;11:44. PubMed PMC

Mottaqi M.S., Mohammadipanah F., Sajedi H. Contribution of machine learning approaches in response to SARS-CoV-2 infection. Inform Med Unlocked. 2021;23 PubMed PMC

Elghamrawy S., Hassanien A.E. Diagnosis and prediction model for COVID-19 patient's response to treatment based on convolutional neural networks and whale optimization algorithm using CT images. 2020. https://doi.org/10.1101/2020.04.16.20063990 medRxiv. DOI

Wang X., Peng Y., Lu L., Lu Z., Bagheri M., Summers R.M. Proceedings of the IEEE conference on computer vision and pattern recognition. 2017. ChestX-ray8: hospital-scale chest X-ray database and benchmarks on weakly-supervised classification and localization of common thorax diseases; pp. 2097–2106.

2023. https://github.com/aimezina/cs_treatment_ai_prediction Source code.

Lundberg S.M., Erion G., Chen H., DeGrave A., Prutkin J.M., Nair B., et al. From local explanations to global understanding with explainable AI for trees. Nat Mach Intell. 2020;2:2522–5839. PubMed PMC

Verploegh I.S., Lazar N.A., Bartels R.H., Volovici V. Evaluation of the use of P values in neurosurgical literature: from statistical significance to clinical irrelevance. World Neurosurg. 2022;161:280–283. PubMed

Ronneberger O., Fischer P., Brox T. International conference on medical image computing and computer-assisted intervention. Springer; 2015. U-Net: convolutional networks for biomedical image segmentation; pp. 234–241.

Mezina A., Burget R. Detection of post-COVID-19-related pulmonary diseases in X-ray images using Vision Transformer-based neural network. Biomed Signal Process Control. 2024;87

Lu B., Luktarhan N., Ding C., Zhang W. ICLSTM: encrypted traffic service identification based on inception-LSTM neural network. Symmetry. 2021;13:1080.

Dong Y., Liu Q., Du B., Zhang L. Weighted feature fusion of convolutional neural network and graph attention network for hyperspectral image classification. IEEE Trans Image Process. 2022;31:1559–1572. PubMed

Dosovitskiy A., Beyer L., Kolesnikov A., Weissenborn D., Zhai X., Unterthiner T., et al. An image is worth 16x16 words: transformers for image recognition at scale. 2020. arXiv:2010.11929 arXiv preprint.

Ben-Baruch E., Ridnik T., Zamir N., Noy A., Friedman I., Protter M., et al. Asymmetric loss for multi-label classification. 2020. arXiv:2009.14119 arXiv preprint.

Lin T.-Y., Goyal P., Girshick R., He K., Dollár P. Proceedings of the IEEE international conference on computer vision. 2017. Focal loss for dense object detection; pp. 2980–2988.

Tan M., Le Q. International conference on machine learning. PMLR; 2019. EfficientNet: rethinking model scaling for convolutional neural networks; pp. 6105–6114.

Simonyan K., Zisserman A. Very deep convolutional networks for large-scale image recognition. 2014. arXiv:1409.1556 arXiv preprint.

Zhou C., Song J., Zhou S., Zhang Z., Xing J. COVID-19 detection based on image regrouping and resnet-SVM using chest X-ray images. IEEE Access. 2021;9:81902–81912. PubMed PMC

Yu Z., Li X., Sun H., Wang J., Zhao T., Chen H., et al. Rapid identification of COVID-19 severity in CT scans through classification of deep features. Biomed Eng Online. 2020;19:1–13. PubMed PMC

Rhys H. Simon and Schuster; 2020. Machine learning with R, the tidyverse, and mlr.

Bhuvaneswari G., Manikandan G. A novel machine learning framework for diagnosing the type 2 diabetics using temporal fuzzy ant miner decision tree classifier with temporal weighted genetic algorithm. Computing. 2018;100:759–772.

Versaci M., Angiulli G., La Foresta F., Crucitti P., Laganá F., Pellicanó D., et al. International conference on applied intelligence and informatics. Springer; 2022. Innovative soft computing techniques for the evaluation of the mechanical stress state of steel plates; pp. 14–28.

Donyatalab Y., Gündoğdu F.K., Farid F., Seyfi-Shishavan S.A., Farrokhizadeh E., Kahraman C. Novel spherical fuzzy distance and similarity measures and their applications to medical diagnosis. Expert Syst Appl. 2022;191