BACKGROUND: This study develops a deep learning-based automated lesion segmentation model for whole-body 3D18F-fluorodeoxyglucose (FDG)-Position emission tomography (PET) with computed tomography (CT) images agnostic to disease location and site. METHOD: A publicly available lesion-annotated dataset of 1014 whole-body FDG-PET/CT images was used to train, validate, and test (70:10:20) eight configurations with 3D U-Net as the backbone architecture. The best-performing model on the test set was further evaluated on 3 different unseen cohorts consisting of osteosarcoma or neuroblastoma (OS cohort) (n = 13), pediatric solid tumors (ST cohort) (n = 14), and adult Pheochromocytoma/Paraganglioma (PHEO cohort) (n = 40). Both lesion-level and patient-level statistical analyses were conducted to validate the performance of the model on different cohorts. RESULTS: The best performing 3D full resolution nnUNet model achieved a lesion-level sensitivity and DISC of 71.70 % and 0.40 for the test set, 97.83 % and 0.73 for ST, 40.15 % and 0.36 for OS, and 78.37 % and 0.50 for the PHEO cohort. For the test set and PHEO cohort, the model has missed small volume and lower uptake lesions (p < 0.01), whereas no statistically significant differences (p > 0.05) were found in the false positive (FP) and false negative lesions volume and uptake for the OS and ST cohort. The predicted total lesion glycolysis is slightly higher than the ground truth because of FP calls, which experts can easily check and reject. CONCLUSION: The developed deep learning-based automated lesion segmentation AI model which utilizes 3D_FullRes configuration of the nnUNet framework showed promising and reliable performance for the whole-body FDG-PET/CT images.

Artificial Intelligence Resource National Cancer Institute National Institute of Health Bethesda MD 20814 USA

Artificial Intelligence Resource National Cancer Institute National Institute of Health Bethesda MD 20814 USA; Molecular Imaging Branch National Cancer Institute National Institute of Health Bethesda MD 20892 USA

Molecular Imaging Branch National Cancer Institute National Institute of Health Bethesda MD 20892 USA

Pediatric Oncology Branch National Cancer Institute National Institute of Health Bethesda MD 20892 USA

Section on Medical Neuroendocrinology National Institute of Child Health and Human Development National Institutes of Health Bethesda 20892 MD USA

Section on Medical Neuroendocrinology National Institute of Child Health and Human Development National Institutes of Health Bethesda 20892 MD USA; AKESO Prague 5 Czech Republic

See more in PubMed

Blanc-Durand P, Jegou S, Kanoun S, Berriolo-Riedinger A, Bodet-Milin C, Kraeber-Bodere F, Carlier T, Le Gouill S, Casasnovas RO, Meignan M, & Itti E (2021). Fully automatic segmentation of diffuse large B cell lymphoma lesions on 3D FDG-PET/CT for total metabolic tumour volume prediction using a convolutional neural network. Eur J Nucl Med Mol Imaging, 48(5), 1362–1370. 10.1007/s00259-020-05080-7 PubMed DOI

Cardoso MJ, Li W, Brown R, Ma N, Kerfoot E, Wang Y, Murrey B, Myronenko A, Zhao C, Yang D, Nath V, He Y, Xu Z, Hatamizadeh A, Myronenko A, Zhu W, Liu Y, Zheng M, Tang Y, … Feng A (2022). MONAI: An open-source framework for deep learning in healthcare. arXiv preprint, arXiv:2211.02701. 10.48550/arXiv.2211.02701 DOI

Chen L-C, Papandreou G, Kokkinos I, Murphy K, & Yuille AL (2018). DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs. IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, 40(4), 15. PubMed

Clark K, Vendt B, Smith K, Freymann J, Kirby J, Koppel P, Moore S, Phillips S, Maffitt D, Pringle M, Tarbox L, & Prior F (2013). The Cancer Imaging Archive (TCIA): maintaining and operating a public information repository. J Digit Imaging, 26(6), 1045–1057. 10.1007/s10278-013-9622-7 PubMed DOI PMC

Constantino CS, Leocadio S, Oliveira FPM, Silva M, Oliveira C, Castanheira JC, Silva A, Vaz S, Teixeira R, Neves M, Lucio P, Joao C, & Costa DC (2023). Evaluation of Semiautomatic and Deep Learning-Based Fully Automatic Segmentation Methods on [(18)F]FDG PET/CT Images from Patients with Lymphoma: Influence on Tumor Characterization. J Digit Imaging, 36(4), 1864–1876. 10.1007/s10278-023-00823-y PubMed DOI PMC

Dice LR (1945). Measures of the Amount of Ecologic Association Between Species. Ecology, 26(3), 297–302. 10.2307/1932409 DOI

Fallahpoor M, Chakraborty S, Pradhan B, Faust O, Barua PD, Chegeni H, & Acharya R (2024). Deep learning techniques in PET/CT imaging: A comprehensive review from sinogram to image space. Comput Methods Programs Biomed, 243, 107880. 10.1016/j.cmpb.2023.107880 PubMed DOI

Ferrandez MC, Golla SSV, Eertink JJ, de Vries BM, Lugtenburg PJ, Wiegers SE, Zwezerijnen GJC, Pieplenbosch S, Kurch L, Huttmann A, Hanoun C, Duhrsen U, de Vet HCW, Petra, Zijlstra JM, & Boellaard R (2023). An artificial intelligence method using FDG PET to predict treatment outcome in diffuse large B cell lymphoma patients. Sci Rep, 13(1), 13111. 10.1038/s41598-023-40218-1 PubMed DOI PMC

Fruh M, Fischer M, Schilling A, Gatidis S, & Hepp T (2021). Weakly supervised segmentation of tumor lesions in PET-CT hybrid imaging. J Med Imaging (Bellingham), 8(5), 054003. 10.1117/1.JMI.8.5.054003 PubMed DOI PMC

Gatidis S, Früh M, Fabritius M, Gu S, Nikolaou K, Fougère CL, Ye J, He J, Peng Y, Bi L, Ma J, Wang B, Zhang J, Huang Y, Heiliger L, Marinov Z, Stiefelhagen R, Egger J, Kleesiek J, … Küstner T (2023). The autoPET challenge: Towards fully automated lesion segmentation in oncologic PET/CT imaging.

Gatidis S, Hepp T, Fruh M, La Fougere C, Nikolaou K, Pfannenberg C, Scholkopf B, Kustner T, Cyran C, & Rubin D (2022). A whole-body FDG-PET/CT Dataset with manually annotated Tumor Lesions. Sci Data, 9(1), 601. 10.1038/s41597-022-01718-3 PubMed DOI PMC

Gatidis S, & Küstner T (2022). A whole-body FDG-PET/CT dataset with manually annotated tumor lesions (FDG-PET-CT-Lesions). 10.7937/gkr0-xv29 PubMed DOI PMC

Giavarina D (2015). Understanding Bland Altman analysis. Biochemia Medica, 25(2), 10. 10.11613/BM.2015.015 PubMed DOI PMC

Hadlich M, Marinov Z, & Stiefelhagen R (2023). AutoPET Challenge 2023: Sliding Window-based Optimization of U-Net. arXiv preprint, 2. 10.48550/arXiv.2309.12114 DOI

Hassan FU, & Cook GJ (2012). PET/CT in oncology. Clinical Medicine, 12(4), 368–372. 10.7861/clinmedicine.12-4-368 PubMed DOI PMC

Hatamizadeh A, Nath V, Tang Y, Yang D, Roth H, & Xu D (2022). Swin UNETR: Swin Transformers for Semantic Segmentation of Brain Tumors in MRI Images. arXiv preprint. 10.48550/arXiv.2201.01266 DOI

Im HJ, Bradshaw T, Solaiyappan M, & Cho SY (2018). Current Methods to Define Metabolic Tumor Volume in Positron Emission Tomography: Which One is Better? Nucl Med Mol Imaging, 52(1), 5–15. 10.1007/s13139-017-0493-6 PubMed DOI PMC

Isensee F, Jaeger PF, Kohl SAA, Petersen J, & Maier-Hein KH (2021). nnU-Net: a self-configuring method for deep learning-based biomedical image segmentation. Nat Methods, 18(2), 203–211. 10.1038/s41592-020-01008-z PubMed DOI

Jemaa S, Fredrickson J, Carano RAD, Nielsen T, de Crespigny A, & Bengtsson T (2020). Tumor Segmentation and Feature Extraction from Whole-Body FDG-PET/CT Using Cascaded 2D and 3D Convolutional Neural Networks. J Digit Imaging, 33(4), 888–894. 10.1007/s10278-020-00341-1 PubMed DOI PMC

Jiang Y, He X, Lee M-LT, Rosner B, & Yan J (2020). Wilcoxon Rank-Based Tests for Clustered Data with R Package clusrank. JournalofStatisticalSoftware, 96(6). 10.18637/jss.v096.i06 DOI

Kaczanowska S, Murty T, Alimadadi A, Contreras CF, Duault C, Subrahmanyam PB, Reynolds W, Gutierrez NA, Baskar R, Wu CJ, Michor F, Altreuter J, Liu Y, Jhaveri A, Duong V, Anbunathan H, Ong C, Zhang H, Moravec R, … Kaplan RN (2024). Immune determinants of CAR-T cell expansion in solid tumor patients receiving GD2 CAR-T cell therapy. Cancer Cell, 42(1), 35–51 e38. 10.1016/j.ccell.2023.11.011 PubMed DOI PMC

Karimdjee M, Delaby G, Huglo D, Baillet C, Willaume A, Dujardin S, & Bailliez A (2023). Evaluation of a convolution neural network for baseline total tumor metabolic volume on [18F]FDG PET in diffuse large B cell lymphoma. European Radiology, 33, 2286–3395. 10.1007/s00330-022-09375-1 PubMed DOI

Koo TK, & Li MY (2016). A Guideline of Selecting and Reporting Intraclass Correlation Coefficients for Reliability Research. Journal of Chiropractic Medicine, 15(2), 9. 10.1016/j.jcm.2016.02.012 PubMed DOI PMC

Landis JR, & Koch GG (1977). The measurement of observer agreement for categorical data. Biometrics, 33(1), 16. 10.2307/2529310 PubMed DOI

Lay N (2023). RandomSplit. https://github.com/nslay/RandomSplit

Leung KH, Rowe SP, Sadaghiani MS, Leal JP, Mena E, Choyke PL, Du Y, & Pomper MG (2024). Deep Semisupervised Transfer Learning for Fully Automated Whole-Body Tumor Quantification and Prognosis of Cancer on PET/CT. J Nucl Med. 10.2967/jnumed.123.267048 PubMed DOI PMC

Liu Z, Lin Y, Cao Y, Hu H, Wei Y, Zhang Z, Lin S, & Guo B (2021). Swin Transformer: Hierarchical Vision Transformer using Shifted Windows. arXiv preprint. 10.48550/arXiv.2103.14030 DOI

Peng Y, Kim J, Feng D, & Bi L (2022). Automatic Tumor Segmentation via False Positive Reduction Network for Whole-Body Multi-Modality PET/CT Images. arXiv preprint. 10.48550/arXiv.2209.07705 DOI

Phil T, Albrecht T, Gay S, & Rasmussen ME (2023). Sikerdebaard/dcmrtstruct2nii: dcmrtstruct2nii v5 (Version v5). In Zenodo.

Ronneberger O, Fischer P, & Brox T (2015). U-Net: Convolutional Networks for Biomedical Image Segmentation. Medical Image Computing and Computer-Assisted Intervention, Pt Iii, 9351, 234–241. 10.1007/978-3-319-24574-4_28 DOI

Sadaghiani MS, Rowe SP, & Sheikhbahaei S (2021). Applications of artificial intelligence in oncologic 18F-FDG PET/CT imaging: a systematic review. Annals of Translational Medicine, 9(9). PubMed PMC

Satoh Y, Nambu A, Ichikawa T, & Onishi H (2014). Whole-body total lesion glycolysis measured on fluorodeoxyglucose positron emission tomography/computed tomography as a prognostic variable in metastatic breast cancer. BMC Cancer, 14(525). 10.1186/1471-2407-14-525 PubMed DOI PMC

Shiri I, Amini M, Yousefirizi F, Sadr AV, Hajianfar G, Salimi Y, Mansouri Z, Jenabi E, Maghsudi M, Mainta I, Becker M, Rahmim A, & Zaidi H (2024). Information fusion for fully automated segmentation of head and neck tumors from PET and CT images. Medical Physics, 51, 319–333. 10.1002/mp.16615 PubMed DOI

Sibille L, Seifert R, Avramovic N, Vehren T, Spottiswoode B, Zuehlsdorff S, & Schäfers M (2020). 18F-FDG PET/CT Uptake Classification in Lymphoma and Lung Cancer by Using Deep Convolutional Neural Networks. Radiology, 294(2), 445–452. 10.1148/radiol.2019191114 PubMed DOI

Sibille L, Zhan X, & Xiang L (2022). Whole-body tumor segmentation of 18F-FDG PET/CT using cascaded and ensembled convolutional neural networks. arXiv preprint. 10.48550/arXiv.2210.08068 DOI

Silversmith W (2021). cc3d: Connected components on multilabel 3D & 2D images. (3.2.1).

Sørensen T (1948). A method of establishing groups of equal amplitude in plant sociology based on similarity of species and its application to analyses of the vegetation on Danish commons (Publication Number 4) https://www.royalacademy.dk/Publications/High/295_S%C3%B8rensen,%20Thorvald.pdf

Tarai S, Lundstrom E, Sjoholm T, Jonsson H, Korenyushkin A, Ahmad N, Pedersen MA, Molin D, Enblad G, Strand R, Ahlstrom H, & Kullberg J (2024). Improved automated tumor segmentation in whole-body 3D scans using multi-directional 2D projection-based priors. Heliyon, 10(4), e26414. 10.1016/j.heliyon.2024.e26414 PubMed DOI PMC

Wang H, Zhang N, Huo L, & Zhang B (2019). Dual-modality multi-atlas segmentation of torso organs from [(18)F]FDG-PET/CT images. Int J Comput Assist Radiol Surg, 14(3), 473–482. 10.1007/s11548-018-1879-3 PubMed DOI

Wang X, Jemaa S, Fredrickson J, Coimbra AF, Nielsen T, De Crespigny A, Bengtsson T, & Carano RAD (2022). Heart and bladder detection and segmentation on FDG PET/CT by deep learning. BMC Med Imaging, 22(1), 58. 10.1186/s12880-022-00785-7 PubMed DOI PMC

Wasserthal J, Breit H-C, Meyer MT, Pradella M, Hinck D, Sauter AW, Heye T, Boll DT, Cyriac J, Yang S, Bach M, & Segeroth M (2023). TotalSegmentator: Robust Segmentation of 104 Anatomic Structures in CT Images. Radiology: Artificial Intelligence, 5(5). 10.1148/ryai.230024 PubMed DOI PMC

Weber WA (2010). Quantitative analysis of PET studies. Radiother Oncol, 96(3), 308–310. 10.1016/j.radonc.2010.07.004 PubMed DOI

Yang E, Kim JH, Min JH, Jeong WK, Hwang JA, Lee JH, Shin J, Kim H, Lee SE, & Baek SY (2024). nnU-Net-Based Pancreas Segmentation and Volume Measurement on CT Imaging in Patients with Pancreatic Cancer. Acad Radiol. 10.1016/j.acra.2024.01.004 PubMed DOI

Ye J, Wang H, Huang Z, Deng Z, Su Y, Tu C, Wu Q, Yang Y, Wei M, Niu J, & He J (2022). Exploring Vanilla U-Net for Lesion Segmentation from Whole-body FDG-PET/CT Scans. arXiv preprint. 10.48550/arXiv.2210.07490 DOI

Yousefirizi F, Klyuzhin IS, O JH, Harsini S, Tie X, Shiri I, Shin M, Lee C, Cho SY, Bradshaw TJ, Zaidi H, Benard F, Sehn LH, Savage KJ, Steidl C, Uribe CF, & Rahmim A (2024). TMTV-Net: fully automated total metabolic tumor volume segmentation in lymphoma PET/CT images - a multi-center generalizability analysis. Eur J Nucl Med Mol Imaging, 51(7), 1937–1954. 10.1007/s00259-024-06616-x PubMed DOI

Zhang G, Bao C, Liu Y, Wang Z, Du L, Zhang Y, Wang F, Xu B, Zhou SK, & Liu R (2023). (18)F-FDG-PET/CT-based deep learning model for fully automated prediction of pathological grading for pancreatic ductal adenocarcinoma before surgery. EJNMMI Res, 13(1), 49. 10.1186/s13550-023-00985-4 PubMed DOI PMC