Heart failure (HF) is one of the leading causes of mortality and hospitalization worldwide. The accurate prediction of mortality and readmission risk provides crucial information for guiding decision making. Unfortunately, traditional predictive models reached modest accuracy in HF populations. We therefore aimed to present predictive models based on machine learning (ML) techniques in HF patients that were externally validated. We searched four databases and the reference lists of the included papers to identify studies in which HF patient data were used to create a predictive model. Literature screening was conducted in Academic Search Ultimate, ERIC, Health Source Nursing/Academic Edition and MEDLINE. The protocol of the current systematic review was registered in the PROSPERO database with the registration number CRD42022344855. We considered all types of outcomes: mortality, rehospitalization, response to treatment and medication adherence. The area under the receiver operating characteristic curve (AUC) was used as the comparator parameter. The literature search yielded 1649 studies, of which 9 were included in the final analysis. The AUCs for the machine learning models ranged from 0.6494 to 0.913 in independent datasets, whereas the AUCs for statistical predictive scores ranged from 0.622 to 0.806. Our study showed an increasing number of ML predictive models concerning HF populations, although external validation remains infrequent. However, our findings revealed that ML approaches can outperform conventional risk scores and may play important role in HF management.

Department of Anthropology Ludwik Hirszfeld Institute of Immunology and Experimental Therapy Polish Academy of Sciences 53 114 Wroclaw Poland

Department of Information and Knowledge Engineering Prague University of Economics and Business W Churchill Sq 1938 4 130 67 Prague Czech Republic

Department of Physiology and Pathophysiology Wroclaw Medical University 50 368 Wroclaw Poland

Department of Psychiatry Division of Consultation Psychiatry and Neuroscience Wroclaw Medical University 50 367 Wroclaw Poland

Institute of Heart Diseases Wroclaw Medical University 50 556 Wroclaw Poland

Institute of Information Systems in Economics Wroclaw University of Economics and Business 53 345 Wroclaw Poland

Zobrazit více v PubMed

Savarese G., Becher P.M., Lund L.H., Seferovic P., Rosano G.M.C., Coats A.J.S. Global burden of heart failure: A comprehensive and updated review of epidemiology. Cardiovasc. Res. 2022:cvac013. doi: 10.1093/cvr/cvac013. PubMed DOI

Ambrosy A.P., Fonarow G.C., Butler J., Chioncel O., Greene S.J., Vaduganathan M., Nodari S., Lam C.S.P., Sato N., Shah A.N., et al. The global health and economic burden of hospitalizations for heart failure: Lessons learned from hospitalized heart failure registries. J. Am. Coll. Cardiol. 2014;63:1123–1133. doi: 10.1016/j.jacc.2013.11.053. PubMed DOI

Gulshan V., Peng L., Coram M., Stumpe M.C., Wu D., Narayanaswamy A., Venugopalan S., Widner K., Madams T., Cuadros J., et al. Development and Validation of a Deep Learning Algorithm for Detection of Diabetic Retinopathy in Retinal Fundus Photographs. JAMA. 2016;316:2402–2410. doi: 10.1001/jama.2016.17216. PubMed DOI

Mohammad M.A., Olesen K.K.W., Koul S., Gale C.P., Rylance R., Jernberg T., Baron T., Spaak J., James S., Lindahl B., et al. Development and validation of an artificial neural network algorithm to predict mortality and admission to hospital for heart failure after myocardial infarction: A nationwide population-based study. Lancet Digit. Health. 2022;4:e37–e45. doi: 10.1016/S2589-7500(21)00228-4. PubMed DOI

Handelman G.S., Kok H.K., Chandra R.V., Razavi A.H., Lee M.J., Asadi H. eDoctor: Machine learning and the future of medicine. J. Intern. Med. 2018;284:603–619. doi: 10.1111/joim.12822. PubMed DOI

Ahmad T., Lund L.H., Rao P., Ghosh R., Warier P., Vaccaro B., Dahlström U., O’Connor C.M., Felker G.M., Desai N.R. Machine Learning Methods Improve Prognostication, Identify Clinically Distinct Phenotypes, and Detect Heterogeneity in Response to Therapy in a Large Cohort of Heart Failure Patients. J. Am. Heart Assoc. 2018;7:e008081. doi: 10.1161/JAHA.117.008081. PubMed DOI PMC

Hosny A., Parmar C., Quackenbush J., Schwartz L.H., Aerts H.J.W.L. Artificial intelligence in radiology. Nat. Rev. Cancer. 2018;18:500–510. doi: 10.1038/s41568-018-0016-5. PubMed DOI PMC

Walsh C.G., Ribeiro J.D., Franklin J.C. Predicting suicide attempts in adolescents with longitudinal clinical data and machine learning. J. Child Psychol. Psychiatry. 2018;59:1261–1270. doi: 10.1111/jcpp.12916. PubMed DOI

Esteva A., Kuprel B., Novoa R.A., Ko J., Swetter S.M., Blau H.M., Thrun S. Dermatologist-level classification of skin cancer with deep neural networks. Nature. 2017;542:115–118. doi: 10.1038/nature21056. PubMed DOI PMC

Motwani M., Dey D., Berman D.S., Germano G., Achenbach S., Al-Mallah M., Andreini D., Budoff M.J., Cademartiri F., Callister T.Q., et al. Machine learning for prediction of all-cause mortality in patients with suspected coronary artery disease: A 5-year multicentre prospective registry analysis. Eur. Heart J. 2016;38:500–507. doi: 10.1093/eurheartj/ehw188. PubMed DOI PMC

Henneges C., Morbach C., Sahiti F., Scholz N., Frantz S., Ertl G., Angermann C.E., Störk S. Sex-Specific Bimodal Clustering ofLeft Ventricular Ejection Fraction in Patients with Acute Heart Failure. ESC Heart Fail. 2021;9:786–790. doi: 10.1002/ehf2.13618. PubMed DOI PMC

Nowak R.M., Reed B.P., DiSomma S., Nanayakkara P., Moyer M., Millis S., Levy P. Presenting Phenotypes of Acute HeartFailure Patients in the ED: Identification and Implications. Am. J. Emerg. Med. 2017;35:536–542. doi: 10.1016/j.ajem.2016.12.003. PubMed DOI

Ahmad T., Desai N., Wilson F., Schulte P., Dunning A., Jacoby D., Allen L., Fiuzat M., Rogers J., Felker G.M., et al. ClinicalImplications of Cluster Analysis-Based Classification of Acute Decompensated Heart Failure and Correlation with BedsideHemodynamic Profiles. PLoS ONE. 2016;11:e0145881. doi: 10.1371/journal.pone.0145881. PubMed DOI PMC

Urban S., Błaziak M., Jura M., Iwanek G., Zdanowicz A., Guzik M., Borkowski A., Gajewski P., Biegus J., Siennicka A., et al. Novel Phenotyping for Acute Heart Failure-Unsupervised Machine Learning-Based Approach. Biomedicines. 2022;10:1514. doi: 10.3390/biomedicines10071514. PubMed DOI PMC

Asyali M.H. Discrimination power of long-term heart rate variability measures; Proceedings of the 25th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (IEEE Cat No03CH37439); Cancun, Mexico. 17–21 September 2003; pp. 200–203.

Melillo P., Fusco R., Sansone M., Bracale M., Pecchia L. Discrimination power of long-term heart rate variability measures for chronic heart failure detection. Med. Biol. Eng. Comput. 2011;49:67–74. doi: 10.1007/s11517-010-0728-5. PubMed DOI

Liu G., Wang L., Wang Q., Zhou G., Wang Y., Jiang Q. A new approach to detect congestive heart failure using short-term heart rate variability measures. PLoS ONE. 2014;9:e93399. doi: 10.1371/journal.pone.0093399. PubMed DOI PMC

Chen W., Liu G., Su S., Jiang Q., Nguyen H. A CHF detection method based on deep learning with RR intervals; Proceedings of the 2017 39th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC); Jeju, Korea. 11–15 July 2017; pp. 3369–3372. PubMed

Chen W., Zheng L., Li K., Wang Q., Liu G., Jiang Q. A Novel and Effective Method for Congestive Heart Failure Detection and Quantification Using Dynamic Heart Rate Variability Measurement. PLoS ONE. 2016;11:e0165304. doi: 10.1371/journal.pone.0165304. PubMed DOI PMC

Cikes M., Sanchez-Martinez S., Claggett B., Duchateau N., Piella G., Butakoff C., Pouleur A.C., Knappe D., Biering-Sørensen T., Kutyifa V., et al. Machine learning-based phenogrouping in heart failure to identify responders to cardiac resynchronization therapy. Eur. J. Heart Fail. 2019;21:74–85. doi: 10.1002/ejhf.1333. PubMed DOI

Feeny A.K., Rickard J., Patel D., Toro S., Trulock K.M., Park C.J., LaBarbera M.A., Varma N., Niebauer M.J., Sinha S., et al. Machine Learning Prediction of Response to Cardiac Resynchronization Therapy. Circ. Arrhythmia Electrophysiol. 2019;12:e007316. doi: 10.1161/CIRCEP.119.007316. PubMed DOI PMC

Schmitz B., De Maria R., Gatsios D., Chrysanthakopoulou T., Landolina M., Gasparini M., Campolo J., Parolini M., Sanzo A., Galimberti P., et al. Identification of genetic markers for treatment success in heart failure patients: Insight from cardiac resynchronization therapy. Circ. Cardiovasc. Genet. 2014;7:760–770. doi: 10.1161/CIRCGENETICS.113.000384. PubMed DOI

Peressutti D., Sinclair M., Bai W., Jackson T., Ruijsink J., Nordsletten D., Asner L., Hadjicharalambous M., Rinaldi C.A., Rueckert D., et al. A framework for combining a motion atlas with non-motion information to learn clinically useful biomarkers: Application to cardiac resynchronisation therapy response prediction. Med. Image Anal. 2017;35:669–684. doi: 10.1016/j.media.2016.10.002. PubMed DOI

Karanasiou G.S., Tripoliti E.E., Papadopoulos T.G., Kalatzis F.G., Goletsis Y., Naka K.K., Bechlioulis A., Errachid A., Fotiadis D.I. Predicting adherence of patients with HF through machine learning techniques. Healthc. Technol. Lett. 2016;3:165–170. doi: 10.1049/htl.2016.0041. PubMed DOI PMC

Dini F.L., Ballo P., Badano L., Barbier P., Chella P., Conti U., De Tommasi S.M., Galderisi M., Ghio S., Magagnini E., et al. Validation of an echo-Doppler decision model to predict left ventricular filling pressure in patients with heart failure independently of ejection fraction. Eur. J. Echocardiogr. 2010;11:703–710. doi: 10.1093/ejechocard/jeq047. PubMed DOI

Graven L.J., Higgins M.K., Reilly C.M., Dunbar S.B. Heart Failure Symptoms Profile Associated with Depressive Symptoms. Clin. Nurs. Res. 2018;29:73–83. doi: 10.1177/1054773818757312. PubMed DOI

Lagu T., Pekow P.S., Shieh M.-S., Stefan M., Pack Q.R., Kashef M.A., Atreya A.R., Valania G., Slawsky M.T., Lindenauer P.K. Validation and Comparison of Seven Mortality Prediction Models for Hospitalized Patients With Acute Decompensated Heart Failure. Circ. Heart Fail. 2016;9:e002912. doi: 10.1161/CIRCHEARTFAILURE.115.002912. PubMed DOI PMC

Pocock S.J., Ariti C.A., Mcmurray J., Maggioni A.P., Køber L., Squire I.B., Swedberg K., Dobson J., Poppe K.K., Whalley G., et al. Predicting survival in heart failure: A risk score based on 39 372 patients from 30 studies. Eur. Heart J. 2012;34:1404–1413. doi: 10.1093/eurheartj/ehs337. PubMed DOI

Mortazavi B.J., Downing N.S., Bucholz E.M., Dharmarajan K., Manhapra A., Li S.-X., Negahban S.N., Krumholz H.M. Analysis of Machine Learning Techniques for Heart Failure Readmissions. Circ. Cardiovasc. Qual. Outcomes. 2016;9:629–640. doi: 10.1161/CIRCOUTCOMES.116.003039. PubMed DOI PMC

Luo W., Phung Q.-D., Tran T., Gupta S., Rana S., Karmakar C., Shilton A., Yearwood J.L., Dimitrova N., Ho T.B., et al. Guidelines for Developing and Reporting Machine Learning Predictive Models in Biomedical Research: A Multidisciplinary View. J. Med Internet Res. 2016;18:e323. doi: 10.2196/jmir.5870. PubMed DOI PMC

Page M.J., McKenzie J.E., Bossuyt P.M., Boutron I., Hoffmann T.C., Mulrow C.D., Shamseer L., Tetzlaff J.F., Akl E.A., Brennan S.E., et al. The PRISMA 2020 statement: An updated guideline for reporting systematic reviews. BMJ. 2021;372:n71. doi: 10.1136/bmj.n71. PubMed DOI PMC

Kmet L.M., Lee R.C., Cook L.S. Standard quality assessment criteria for Evaluating Primary Research Papers from a Variety of Fields. HTA Initiat. 2004;13:4.

Luo C., Zhu Y., Zhu Z., Li R., Chen G., Wang Z. A machine learning-based risk stratification tool for in-hospital mortality of intensive care unit patients with heart failure. J. Transl. Med. 2022;20:1–9. doi: 10.1186/s12967-022-03340-8. PubMed DOI PMC

Adler E.D., Voors A.A., Klein L., Macheret F., Braun O.O., Urey M.A., Zhu W., Sama I., Tadel M., Campagnari C., et al. Improving risk prediction in heart failure using machine learning. Eur. J. Heart Fail. 2019;22:139–147. doi: 10.1002/ejhf.1628. PubMed DOI

Kwon J.-M., Kim K.-H., Jeon K.-H., Lee S.E., Lee H.-Y., Cho H.-J., Choi J.O., Jeon E.-S., Kim M.-S., Kim J.-J., et al. Artificial intelligence algorithm for predicting mortality of patients with acute heart failure. PLoS ONE. 2019;14:e0219302. doi: 10.1371/journal.pone.0219302. PubMed DOI PMC

Jing L., Cerna A.E.U., Good C.W., Sauers N.M., Schneider G., Hartzel D.N., Leader J.B., Kirchner H.L., Hu Y., Riviello D.M., et al. A Machine Learning Approach to Management of Heart Failure Populations. JACC Heart Fail. 2020;8:578–587. doi: 10.1016/j.jchf.2020.01.012. PubMed DOI

Chirinos J.A., Orlenko A., Zhao L., Basso M.D., Cvijic M.E., Li Z., Spires T.E., Yarde M., Wang Z., Seiffert D.A., et al. Multiple Plasma Biomarkers for Risk Stratification in Patients With Heart Failure and Preserved Ejection Fraction. J. Am. Coll. Cardiol. 2020;75:1281–1295. doi: 10.1016/j.jacc.2019.12.069. PubMed DOI PMC

Kwon J.M., Kim K.H., Jeon K.H., Park J. Deep learning for predicting in-hospital mortality among heart disease patients based on echocardiography. Echocardiography. 2019;36:213–218. doi: 10.1111/echo.14220. PubMed DOI

Mahajan S.M., Ghani R. Using ensemble machine learning methods for predicting risk of readmission for heart failure. Stud. Health Technol. Inform. 2019;264:243–247. PubMed

Mahajan S.M., Ghani R. Combining structured and unstructured data for predicting risk of readmission for heart failure patients. Stud. Health Technol. Inform. 2019;264:238–242. PubMed

Kakarmath S., Golas S., Felsted J., Kvedar J., Jethwani K., Agboola S. Validating a Machine Learning Algorithm to Predict 30-Day Re-Admissions in Patients With Heart Failure: Protocol for a Prospective Cohort Study. JMIR Res. Protoc. 2018;7:e176. doi: 10.2196/resprot.9466. PubMed DOI PMC

Fawcett T. An introduction to ROC analysis. Pattern Recognit. Lett. 2006;27:861–874. doi: 10.1016/j.patrec.2005.10.010. DOI

Hosmer D.W., Lemeshow S., Sturdivant R.X. Applied Logistic Regression. 3rd ed. John Wiley & Sons, Inc.; Hoboken, NJ, USA: 2013. pp. 1–510. DOI

Understanding AUC—ROC Curve|by Sarang Narkhede|Towards Data Science. [(accessed on 24 July 2022)]. Available online: https://towardsdatascience.com/understanding-auc-roc-curve-68b2303cc9c5.

Januzzi J.L., Jr., Sakhuja R., O’Donoghue M., Baggish A.L., Anwaruddin S., Chae C.U., Cameron R., Krauser D.G., Tung R., Camargo C.A., Jr., et al. Utility of Amino-Terminal Pro–Brain Natriuretic Peptide Testing for Prediction of 1-Year Mortality in Patients With Dyspnea Treated in the Emergency Department. Arch. Intern. Med. 2006;166:315–320. doi: 10.1001/archinte.166.3.315. PubMed DOI

McKie P.M., Cataliotti A., Lahr B.D., Martin F.L., Redfield M.M., Bailey K.R., Rodeheffer R.J., Burnett J.C. The Prognostic Value of N-Terminal Pro–B-Type Natriuretic Peptide for Death and Cardiovascular Events in Healthy Normal and Stage A/B Heart Failure Subjects. J. Am. Coll. Cardiol. 2010;55:2140–2147. doi: 10.1016/j.jacc.2010.01.031. PubMed DOI PMC

Royston P., Altman D.G. External validation of a Cox prognostic model: Principles and methods. BMC Med. Res. Methodol. 2013;13:33. doi: 10.1186/1471-2288-13-33. PubMed DOI PMC

Siontis G.C.M., Tzoulaki I., Castaldi P.J., Ioannidis J.P.A. External validation of new risk prediction models is infrequent and reveals worse prognostic discrimination. J. Clin. Epidemiol. 2015;68:25–34. PubMed

Lemeshow S., Klar J., Teres D., Avrunin J.S., Gehlbach S.H., Rapoport J., Rué M. Mortality probability models for patients in the intensive care unit for 48 or 72 hours: A prospective, multicenter study. Crit. Care Med. 1994;22:1351–1358. doi: 10.1097/00003246-199409000-00003. PubMed DOI

Adrie C., Francais A., Alvarez-Gonzalez A., Mounier R., Azoulay E., Zahar J.-R., Clec’H C., Goldgran-Toledano D., Hammer L., Descorps-Declere A., et al. Model for predicting short-term mortality of severe sepsis. Crit. Care. 2009;13:R72. doi: 10.1186/cc7881. PubMed DOI PMC

Agbor V.N., Essouma M., Ntusi N.A., Nyaga U.F., Bigna J.J., Noubiap J.J. Heart failure in sub-Saharan Africa: A contemporaneous systematic review and meta-analysis. Int. J. Cardiol. 2018;257:207–215. doi: 10.1016/j.ijcard.2017.12.048. PubMed DOI

Bahrami H., Kronmal R., Bluemke D.A., Olson J., Shea S., Liu K., Burke G.L., Lima J.A. Differences in the incidence of congestive heart failure by ethnicity: The multi-ethnic study of atherosclerosis. Arch. Intern. Med. 2008;168:2138–2145. doi: 10.1001/archinte.168.19.2138. PubMed DOI PMC

Bazoukis G., Stavrakis S., Zhou J., Bollepalli S.C., Tse G., Zhang Q., Singh J.P., Armoundas A.A. Machine learning versus conventional clinical methods in guiding management of heart failure patients—A systematic review. Heart Fail. Rev. 2020;26:23–34. doi: 10.1007/s10741-020-10007-3. PubMed DOI PMC

Ribeiro M.T., Singh S., Guestrin C. “Why should i trust you?” explaining the predictions of any classifier; Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining; San Francisco, CA USA. 13–17 August 2016; pp. 1135–1144.

Lundberg S.M., Erion G., Chen H., DeGrave A., Prutkin J.M., Nair B., Katz R., Himmelfarb J., Bansal N., Lee S.-I. From local explanations to global understanding with explainable ai for trees. Nat. Mach. Intell. 2020;2:2522–5839. doi: 10.1038/s42256-019-0138-9. PubMed DOI PMC

Duchnowski P., Hryniewiecki T., Koźma M., Mariusz K., Piotr S. High-sensitivity troponin T is a prognostic marker of hemodynamic instability in patients undergoing valve surgery. Biomark Med. 2018;12:1303–1309. doi: 10.2217/bmm-2018-0186. PubMed DOI

Duchnowski P., Hryniewiecki T., Kuśmierczyk M., Szymański P. Postoperative high-sensitivity troponin T as a predictor of sudden cardiac arrest in patients undergoing cardiac surgery. Cardiol. J. 2019;26:777–781. doi: 10.5603/CJ.a2019.0005. PubMed DOI PMC

Cacciatore F., Abete P., Mazzella F., Furgi G., Nicolino A., Longobardi G., Testa G., Langellotto A., Infante T., Napoli C., et al. Six-minute walking test but not ejection fraction predicts mortality in elderly patients undergoing cardiac rehabilitation following coronary artery bypass grafting. Eur. J. Prev. Cardiol. 2011;19:1401–1409. doi: 10.1177/1741826711422991. PubMed DOI

Briongos-Figuero S., Estévez A., Pérez M.L., Martínez-Ferrer J.B., García E., Viñolas X., Arenal Á., Alzueta J., Muñoz-Aguilera R. Prognostic role of NYHA class in heart failure patients undergoing primary prevention ICD therapy. ESC Heart Fail. 2019;7:280–284. doi: 10.1002/ehf2.12548. PubMed DOI PMC

Al-Tamimi M.A.A., Gillani S.W., Abd Alhakam M.E., Sam K.G. Factors Associated with Hospital Readmission of Heart Failure Patients. Front. Pharmacol. 2021;12:732760. PubMed PMC

Baert A., Clays E., Bolliger L., De Smedt D., Lustrek M., Vodopija A., Bohanec M., Puddu P.E., Ciancarelli M.C., Schiariti M., et al. A Personal Decision Support System for Heart Failure Management (HeartMan): Study protocol of the HeartMan randomized controlled trial. BMC Cardiovasc. Disord. 2018;18:186. doi: 10.1186/s12872-018-0921-2. PubMed DOI PMC

Sengupta P.P., Shrestha S., Kagiyama N., Hamirani Y., Kulkarni H., Yanamala N., Bing R., Chin C.W.L., Pawade T.A., Messika-Zeitoun D., et al. A Machine-Learning Framework to Identify Distinct Phenotypes of Aortic Stenosis Severity. JACC Cardiovasc. Imaging. 2021;14:1707–1720. doi: 10.1016/j.jcmg.2021.03.020. PubMed DOI PMC

Attia Z.I., Noseworthy P.A., Lopez-Jimenez F., Asirvatham S.J., Deshmukh A.J., Gersh B.J., Carter R.E., Yao X., Rabinstein A.A., Erickson B.J., et al. An artificial intelligence-enabled ECG algorithm for the identification of patients with atrial fibrillation during sinus rhythm: A retrospective analysis of outcome prediction. Lancet. 2019;394:861–867. doi: 10.1016/S0140-6736(19)31721-0. PubMed DOI

Ghazi L., Ahmad T., Wilson F.P. A Clinical Framework for Evaluating Machine Learning Studies. JACC Heart Fail. 2022 doi: 10.1016/j.jchf.2022.07.002. in press. PubMed DOI

Machine Learning Approach to Understand Worsening Renal Function in Acute Heart Failure