This study aims to develop a fully automated imaging protocol independent system for pituitary adenoma segmentation from magnetic resonance imaging (MRI) scans that can work without user interaction and evaluate its accuracy and utility for clinical applications. We trained two independent artificial neural networks on MRI scans of 394 patients. The scans were acquired according to various imaging protocols over the course of 11 years on 1.5T and 3T MRI systems. The segmentation model assigned a class label to each input pixel (pituitary adenoma, internal carotid artery, normal pituitary gland, background). The slice segmentation model classified slices as clinically relevant (structures of interest in slice) or irrelevant (anterior or posterior to sella turcica). We used MRI data of another 99 patients to evaluate the performance of the model during training. We validated the model on a prospective cohort of 28 patients, Dice coefficients of 0.910, 0.719, and 0.240 for tumour, internal carotid artery, and normal gland labels, respectively, were achieved. The slice selection model achieved 82.5% accuracy, 88.7% sensitivity, 76.7% specificity, and an AUC of 0.904. A human expert rated 71.4% of the segmentation results as accurate, 21.4% as slightly inaccurate, and 7.1% as coarsely inaccurate. Our model achieved good results comparable with recent works of other authors on the largest dataset to date and generalized well for various imaging protocols. We discussed future clinical applications, and their considerations. Models and frameworks for clinical use have yet to be developed and evaluated.

1st Faculty of Medicine Charles University Prague Kateřinská 1660 32 121 08 Praha 2 Czech Republic

3rd Faculty of Medicine Charles University Prague Ruská 87 100 00 Praha 10 Czech Republic

Department of Cybernetics Faculty of Electrical Engineering Czech Technical University Prague Technická 2 166 27 Praha 6 Czech Republic

Department of Neurosurgery and Neurooncology 1st Faculty of Medicine Charles University Central Military Hospital Prague U Vojenské nemocnice 1200 169 02 Praha 6 Czech Republic

Department of Neurosurgery Faculty of Medicine University of Ostrava University Hospital Ostrava 17 listopadu 1790 5 708 52 Ostrava Poruba Czech Republic

Department of Radiodiagnostics Central Military Hospital Prague U Vojenské nemocnice 1200 169 02 Praha 6 Czech Republic

See more in PubMed

Daly AF, Beckers A (2020) The epidemiology of pituitary adenomas. Endocrinol Metab Clin North Am 49(3):347–355. https://doi.org/10.1016/j.ecl.2020.04.002 PubMed DOI

Molitch ME (2017) Diagnosis and treatment of pituitary adenomas: a review. JAMA 317(5):516–524. https://doi.org/10.1001/jama.2016.19699 PubMed DOI

Celtikci E (2018) A systematic review on machine learning in neurosurgery: the future of decision-making in patient care. Turk Neurosurg 28(2):167–173. https://doi.org/10.5137/1019-5149.JTN.20059-17.1 PubMed DOI

Dai C, Sun B, Wang R, Kang J (2021) The application of artificial intelligence and machine learning in pituitary adenomas. Front Oncol 11:784819. https://doi.org/10.3389/fonc.2021.784819 PubMed DOI PMC

Wang H, Zhang W, Li S, Fan Y, Feng M, Wang R (2021) Development and evaluation of deep learning-based automated segmentation of pituitary adenoma in clinical task. J Clin Endocrinol Metab 106(9):2535–2546. https://doi.org/10.1210/clinem/dgab371 PubMed DOI

Egger J, Zukić D, Freisleben B, Kolb A, Nimsky C (2013) Segmentation of pituitary adenoma: a graph-based method vs. a balloon inflation method. Comput Methods Programs Biomed 110(3):268–278. https://doi.org/10.1016/j.cmpb.2012.11.010 PubMed DOI

Egger J, Kapur T, Nimsky C, Kikinis R (2012) Pituitary adenoma volumetry with 3D Slicer. PloS One 7(12):e51788. https://doi.org/10.1371/journal.pone.0051788 PubMed DOI PMC

Shu X, Zhou Y, Li F, Zhou T, Meng X, Wang F, Zhang Z, Pu J, Xu B (2021) Three-dimensional semantic segmentation of pituitary adenomas based on the deep learning framework-nnU-Net: a clinical perspective. Micromachines 12(12):1473. https://doi.org/10.3390/mi12121473 PubMed DOI PMC

Voglis S, van Niftrik C, Staartjes VE, Brandi G, Tschopp O, Regli L, Serra C (2020) Feasibility of machine learning based predictive modelling of postoperative hyponatremia after pituitary surgery. Pituitary 23(5):543–551. https://doi.org/10.1007/s11102-020-01056-w PubMed DOI

Laws ER, Catalino MP (2020) Editorial. Machine learning and artificial intelligence applied to the diagnosis and management of Cushing disease. Neurosurg Focus 48(6):E6. https://doi.org/10.3171/2020.3.FOCUS20213 PubMed DOI

Fan Y, Hua M, Mou A, Wu M, Liu X, Bao X, Wang R, Feng M (2019) Preoperative noninvasive radiomics approach predicts tumor consistency in patients with acromegaly: development and multicenter prospective validation. Front Endocrinol 10:403. https://doi.org/10.3389/fendo.2019.00403 DOI

Zeynalova A, Kocak B, Durmaz ES, Comunoglu N, Ozcan K, Ozcan G, Turk O, Tanriover N, Kocer N, Kizilkilic O, Islak C (2019) Preoperative evaluation of tumour consistency in pituitary macroadenomas: a machine learning-based histogram analysis on conventional T2-weighted MRI. Neuroradiology 61(7):767–774. https://doi.org/10.1007/s00234-019-02211-2 PubMed DOI

Zhu H, Fang Q, Huang Y, Xu K (2020) Semi-supervised method for image texture classification of pituitary tumors via CycleGAN and optimized feature extraction. BMC Med Inform Decis Mak 20(1):215. https://doi.org/10.1186/s12911-020-01230-x PubMed DOI PMC

Meng T, Guo X, Lian W, Deng K, Gao L, Wang Z, Huang J, Wang X, Long X, Xing B (2020) Identifying facial features and predicting patients of acromegaly using three-dimensional imaging techniques and machine learning. Front Endocrinol 11:492. https://doi.org/10.3389/fendo.2020.00492 DOI

Wei R, Jiang C, Gao J, Xu P, Zhang D, Sun Z, Liu X, Deng K, Bao X, Sun G, Yao Y, Lu L, Zhu H, Wang R, Feng M (2020) Deep-learning approach to automatic identification of facial anomalies in endocrine disorders. Neuroendocrinology 110(5):328–337. https://doi.org/10.1159/000502211 PubMed DOI

Jarrett D, Stride E, Vallis K, Gooding MJ (2019) Applications and limitations of machine learning in radiation oncology. Br J Radiol 92(1100):20190001. https://doi.org/10.1259/bjr.20190001 PubMed DOI PMC

Bong JH, Song HJ, Oh Y, Park N, Kim H, Park S (2018) Endoscopic navigation system with extended field of view using augmented reality technology. Int J Med Robot Comput Assist Surg 14(2). https://doi.org/10.1002/rcs.1886

Yu YL, Yang YJ, Lin C, Hsieh CC, Li CZ, Feng SW, Tang CT, Chung TT, Ma HI, Chen YH, Ju DT, Hueng DY (2017) Analysis of volumetric response of pituitary adenomas receiving adjuvant CyberKnife stereotactic radiosurgery with the application of an exponential fitting model. Medicine 96(4):e4662. https://doi.org/10.1097/MD.0000000000004662 PubMed DOI PMC

Knosp E, Steiner E, Kitz K, Matula C (1993) Pituitary adenomas with invasion of the cavernous sinus space: a magnetic resonance imaging classification compared with surgical findings. Neurosurgery 33(4):610–618. https://doi.org/10.1227/00006123-199310000-00008 PubMed DOI

Micko AS, Wöhrer A, Wolfsberger S, Knosp E (2015) Invasion of the cavernous sinus space in pituitary adenomas: endoscopic verification and its correlation with an MRI-based classification. J Neurosurg 122(4):803–811. https://doi.org/10.3171/2014.12.JNS141083 PubMed DOI

Araujo-Castro M, Pascual-Corrales E, Martínez-Vaello V, Baonza Saiz G, Quiñones de Silva J, Acitores Cancela A, García Cano AM, Rodríguez Berrocal V (2021) Predictive model of surgical remission in acromegaly: age, presurgical GH levels and Knosp grade as the best predictors of surgical remission. J Endocrinol Invest 44(1):183–193. https://doi.org/10.1007/s40618-020-01296-4 PubMed DOI

Hardy J, Vezina JL (1976) Transsphenoidal neurosurgery of intracranial neoplasm. Adv Neurol 15:261–273 PubMed

Wilson G (1979) Neurosurgical management of large and invasive pituitary tumors. Clin Manag Pituit Disord:335–342

Kelly CJ, Karthikesalingam A, Suleyman M, Corrado G, King D (2019) Key challenges for delivering clinical impact with artificial intelligence. BMC Med 17(1):195. https://doi.org/10.1186/s12916-019-1426-2 PubMed DOI PMC

Egger J, Bauer MH, Kuhnt D, Freisleben B, Nimsky C (2011) Pituitary adenoma segmentation. arXiv preprint arXiv:1103.1778. https://doi.org/10.48550/arXiv.1103.1778 DOI

Yushkevich PA, Piven J, Hazlett HC, Smith RG, Ho S, Gee JC, Gerig G (2006) User-guided 3D active contour segmentation of anatomical structures: significantly improved efficiency and reliability. NeuroImage 31(3):1116–1128. https://doi.org/10.1016/j.neuroimage.2006.01.015 PubMed DOI

van Rossum G (1995) Python reference manual. Department of Computer Science [CS] R 9525

Beare R, Lowekamp B, Yaniv Z (2018) Image segmentation, registration and characterization in R with SimpleITK. J Stat Softw 86:8. https://doi.org/10.18637/jss.v086.i08 PubMed DOI PMC

Yaniv Z, Lowekamp BC, Johnson HJ, Beare R (2018) SimpleITK image-analysis notebooks: a collaborative environment for education and reproducible research. J Digit Imaging 31(3):290–303. https://doi.org/10.1007/s10278-017-0037-8 PubMed DOI

Lowekamp BC, Chen DT, Ibáñez L, Blezek D (2013) The design of SimpleITK. Front Neuroinform 7:45. https://doi.org/10.3389/fninf.2013.00045 PubMed DOI PMC

Chollet F (2015) Keras. GitHub Retrieved from: https.github.com/fchollet/keras

Abadi M, Agarwal A, Barham P, Brevdo E, Chen Z, Citro C et al (2016) Tensorflow: Large-scale machine learning on heterogeneous distributed systems. arXiv preprint arXiv:1603.04467

Cerny M (2022) Fully automated imaging protocol independent system for pituitary adenoma segmentation. GitHub repository https://github.com/DrMartinCerny/pituitary_adenoma_segmentation

Ronneberger O, Fischer P, Brox T (2015) U-net: convolutional networks for biomedical image segmentation. In: International Conference on Medical image computing and computer-assisted intervention, pp 234–241. https://doi.org/10.1007/978-3-319-24574-4_28 DOI

Nair V, Hinton GE (2010) Rectified linear units improve restricted boltzmann machines. In: Icml

Shorten C, Khoshgoftaar TM (2019) A survey on image data augmentation for deep learning. J Big Data 6:60. https://doi.org/10.1186/s40537-019-0197-0 DOI

Kingma DP, Ba J (2014) Adam: a method for stochastic optimization. arXiv preprint arXiv:1412.6980

Zou KH, Warfield SK, Bharatha A, Tempany CM, Kaus MR, Haker SJ et al (2004) Statistical validation of image segmentation quality based on a spatial overlap index1: scientific reports. Acad Radiol 11(2):178–189 PubMed DOI PMC

Egger J, Bauer MH, Kuhnt D, Carl B, Kappus C, Freisleben B, Nimsky C (2010) Nugget-cut: a segmentation scheme for spherically-and elliptically-shaped 3D objects. In: Joint Pattern Recognition Symposium. Springer, Berlin, Heidelberg, pp 373–382 DOI

Ikonomakis N, Plataniotis KN, Venetsanopoulos AN (2000) Color image segmentation for multimedia applications. J Intell Robot Syst 28(1):5–20 DOI

Zukić D, Egger J, Bauer MH, Kuhnt D, Carl B, Freisleben B et al (2011) Glioblastoma multiforme segmentation in MRI data with a balloon inflation approach. arXiv preprint arXiv:1102.0634

Takikawa T, Acuna D, Jampani V, Fidler S (2019) Gated-scnn: gated shape cnns for semantic segmentation. In: Proceedings of the IEEE/CVF international conference on computer vision, pp 5229–5238

Isensee F, Jaeger PF, Kohl SA, Petersen J, Maier-Hein KH (2021) nnU-Net: a self-configuring method for deep learning-based biomedical image segmentation. Nat Methods 18(2):203–211 PubMed DOI

Zhang L, Wang X, Yang D, Sanford T, Harmon S, Turkbey B et al (2020) Generalizing deep learning for medical image segmentation to unseen domains via deep stacked transformation. IEEE Trans Med Imaging 39(7):2531–2540 PubMed DOI PMC

Bokhorst JM, Pinckaers H, van Zwam P, Nagtegaal I, van der Laak J, Ciompi F (2018) Learning from sparsely annotated data for semantic segmentation in histopathology images. In: International Conference on Medical Imaging with Deep Learning--Full Paper Track

Zhang Z, Li J, Zhong Z, Jiao Z, Gao X (2019) A sparse annotation strategy based on attention-guided active learning for 3D medical image segmentation. arXiv preprint arXiv:1906.07367

Guo Z, Li X, Huang H, Guo N, Li Q (2019) Deep learning-based image segmentation on multimodal medical imaging. IEEE Trans Radiat Plasma Med Sci 3(2):162–169 PubMed DOI PMC

Le MH, Chen J, Wang L, Wang Z, Liu W, Cheng KTT, Yang X (2017) Automated diagnosis of prostate cancer in multi-parametric MRI based on multimodal convolutional neural networks. Phys Med Biol 62(16):6497 PubMed DOI

Wang J, Berger D, Mattie D & Levman J (2021) Multichannel input pixelwise regression 3D U-Nets for medical image estimation with 3 applications in brain MRI

Zhou T, Ruan S, Canu S (2019) A review: deep learning for medical image segmentation using multi-modality fusion. Array 3:100004 DOI

Pereira S, Pinto A, Alves V, Silva CA (2016) Brain tumor segmentation using convolutional neural networks in MRI images. IEEE Trans Med Imaging 35(5):1240–1251 PubMed DOI

Isensee F, Kickingereder P, Wick W, Bendszus M, Maier-Hein KH (2017) Brain tumor segmentation and radiomics survival prediction: contribution to the brats 2017 challenge. In: International MICCAI Brainlesion Workshop. Springer, Cham, pp 287–297

Isensee F, Kickingereder P, Wick W, Bendszus M, Maier-Hein KH (2018) No new-net. In: International MICCAI Brainlesion Workshop. Springer, Cham, pp 234–244

Cui S, Mao L, Jiang J, Liu C, Xiong S (2018) Automatic semantic segmentation of brain gliomas from MRI images using a deep cascaded neural network. J Healthc Eng 2018:4940593. https://doi.org/10.1155/2018/4940593 PubMed DOI PMC

Wang G, Li W, Ourselin S, Vercauteren T (2017) Automatic brain tumor segmentation using cascaded anisotropic convolutional neural networks. In: International MICCAI brainlesion workshop. Springer, Cham, pp 178–190

Kamnitsas K, Ledig C, Newcombe V, Simpson JP, Kane AD, Menon DK, Rueckert D, Glocker B (2017) Efficient multi-scale 3D CNN with fully connected CRF for accurate brain lesion segmentation. Med Image Anal 36:61–78. https://doi.org/10.1016/j.media.2016.10.004 PubMed DOI

Zhao X, Wu Y, Song G, Li Z, Zhang Y, Fan Y (2018) A deep learning model integrating FCNNs and CRFs for brain tumor segmentation. Med Image Anal 43:98–111. https://doi.org/10.1016/j.media.2017.10.002 PubMed DOI

Myronenko A (2018) 3D MRI brain tumor segmentation using autoencoder regularization. In: International MICCAI Brainlesion Workshop. Springer, Cham, pp 311–320

Clèrigues A, Valverde S, Bernal J, Freixenet J, Oliver A, Lladó X (2020) Acute and sub-acute stroke lesion segmentation from multimodal MRI. Comput Methods Programs Biomed 194:105521 PubMed DOI

Chen L, Wu Y, DSouza AM, Abidin AZ, Wismüller A, Xu C (2018) MRI tumor segmentation with densely connected 3D CNN. In: Medical Imaging 2018: Image Processing, vol 10574. SPIE, pp 357–364

Dolz J, Desrosiers C, Ben Ayed I (2018) IVD-Net: Intervertebral disc localization and segmentation in MRI with a multi-modal UNet. In: International workshop and challenge on computational methods and clinical applications for spine imaging. Springer, Cham, pp 130–143

Dolz J, Gopinath K, Yuan J, Lombaert H, Desrosiers C, Ben Ayed I (2019) HyperDense-Net: a hyper-densely connected CNN for multi-modal image segmentation. IEEE Trans Med Imaging 38(5):1116–1126. https://doi.org/10.1109/TMI.2018.2878669 PubMed DOI

Rokach L (2010) Ensemble-based classifiers. Artif Intell Rev 33(1):1–39 DOI

Nie D, Wang L, Adeli E, Lao C, Lin W, Shen D (2019) 3-D fully convolutional networks for multimodal isointense infant brain image segmentation. IEEE Trans Cybern 49(3):1123–1136. https://doi.org/10.1109/TCYB.2018.2797905 PubMed DOI

Castro DG, Cecílio SA, Canteras MM (2010) Radiosurgery for pituitary adenomas: evaluation of its efficacy and safety. Radiat Oncol 5:109. https://doi.org/10.1186/1748-717X-5-109 PubMed DOI PMC

Girkin CA, Comey CH, Lunsford LD, Goodman ML, Kline LB (1997) Radiation optic neuropathy after stereotactic radiosurgery. Ophthalmology 104(10):1634–1643. https://doi.org/10.1016/s0161-6420(97)30084-0 PubMed DOI