Angiogenesis is the process of new blood vessels growing from existing vasculature. Visualizing them as a three-dimensional (3D) model is a challenging, yet relevant, task as it would be of great help to researchers, pathologists, and medical doctors. A branching analysis on the 3D model would further facilitate research and diagnostic purposes. In this paper, a pipeline of vision algorithms is elaborated to visualize and analyze blood vessels in 3D from formalin-fixed paraffin-embedded (FFPE) granulation tissue sections with two different staining methods. First, a U-net neural network is used to segment blood vessels from the tissues. Second, image registration is used to align the consecutive images. Coarse registration using an image-intensity optimization technique, followed by finetuning using a neural network based on Spatial Transformers, results in an excellent alignment of images. Lastly, the corresponding segmented masks depicting the blood vessels are aligned and interpolated using the results of the image registration, resulting in a visualized 3D model. Additionally, a skeletonization algorithm is used to analyze the branching characteristics of the 3D vascular model. In summary, computer vision and deep learning is used to reconstruct, visualize and analyze a 3D vascular model from a set of parallel tissue samples. Our technique opens innovative perspectives in the pathophysiological understanding of vascular morphogenesis under different pathophysiological conditions and its potential diagnostic role.

Biomedical Center Faculty of Medicine in Pilsen Charles University 32300 Pilsen Czech Republic

Center for Research and Formation in Artificial Intelligence Universidad de Los Andes 111711 Bogota Colombia

Central Biobank Regensburg University and University Hospital Regensburg 93053 Regensburg Germany

Department of Cardiology University Medical Centre Johannes Gutenberg University of Mainz 55131 Mainz Germany

Department of Dermatology University Medical Centre Regensburg 93053 Regensburg Germany

Department of Internal Medicine 2 Division of Hepatology Medical Faculty Mannheim Heidelberg University 68167 Mannheim Germany

Department of Internal Medicine 2 University Hospital Regensburg 93053 Regensburg Germany

Department of Thoracic Surgery Kerckhoff Clinic 61231 Bad Nauheim Germany

Department of Trauma Surgery University Medical Centre Regensburg 93053 Regensburg Germany

Institute of Pathology University Medical Centre Mannheim Heidelberg University 68167 Mannheim Germany

Institute of Pathology University of Regensburg 93053 Regensburg Germany

Zobrazit více v PubMed

Honnegowda T.M., Kumar P., Udupa E.G., Kumar S., Kumar U., Rao P. Role of angiogenesis and angiogenic factors in acute and chronic wound healing. Plast. Aesthetic Res. 2015;2:239–242. doi: 10.4103/2347-9264.165438. DOI

Folkman J. Role of angiogenesis in tumor growth and metastasis. Semin. Oncol. 2002;29:15–18. doi: 10.1053/sonc.2002.37263. PubMed DOI

Hanahan D. Hallmarks of Cancer: New Dimensions. Cancer Discov. 2022;12:31–46. doi: 10.1158/2159-8290.CD-21-1059. PubMed DOI

Guerra A., Belinha J., Jorge R.N. Modelling skin wound healing angiogenesis: A review. J. Theor. Biol. 2018;459:1–17. doi: 10.1016/j.jtbi.2018.09.020. PubMed DOI

Pollefeys M., Koch R., Vergauwen M., Van Gool L. Automated reconstruction of 3D scenes from sequences of images. ISPRS J. Photogramm. Remote Sens. 2001;55:251–267. doi: 10.1016/S0924-2716(00)00023-X. DOI

Gallo A., Muzzupappa M., Bruno F. 3D reconstruction of small sized objects from a sequence of multi-focused images. J. Cult. Herit. 2014;15:173–182. doi: 10.1016/j.culher.2013.04.009. DOI

Carlbom I., Terzopoulos D., Harris K.M. Computer-assisted registration, segmentation, and 3D reconstruction from images of neuronal tissue sections. IEEE Trans. Med. Imaging. 1994;13:351–362. doi: 10.1109/42.293928. PubMed DOI

Tom M., Ramakrishnan V., van Oterendorp C., Deserno T. Medical Imaging 2015: Computer-Aided Diagnosis. SPIE; Bellingham, WA, USA: 2015. Automated Detection of Schlemm’s Canal in Spectral-Domain Optical Coherence Tomography.

Meiburger K.M., Nam S.Y., Chung E., Suggs L.J., Emelianov S.Y., Molinari F. Skeletonization algorithm-based blood vessel quantification using in vivo 3D photoacoustic imaging. Phys. Med. Biol. 2016;61:7994–8009. doi: 10.1088/0031-9155/61/22/7994. PubMed DOI

Toriwaki J.-i., Mori K. Digital and Image Geometry: Advanced Lectures. Springer; Berlin/Heidelberg, Germany: 2002. Distance transformation and skeletonization of 3D pictures and their applications to medical images; pp. 412–429.

Swedlow J. Open Microscopy Environment: OME Is a Consortium of Universities, Research Labs, Industry and Developers Producing Open-Source Software and Format Standards for Microscopy Data. 2020. [(accessed on 20 April 2023)]. Available online: https://discovery.dundee.ac.uk/en/publications/open-microscopy-environment-ome-is-a-consortium-of-universities-r.

Otsu N. A Threshold Selection Method from Gray-Level Histograms. IEEE Trans. Syst. Man Cybern. 1979;9:62–66. doi: 10.1109/TSMC.1979.4310076. DOI

Li Y., Wu H. A Clustering Method Based on K-Means Algorithm. Phys. Procedia. 2012;25:1104–1109. doi: 10.1016/j.phpro.2012.03.206. DOI

Pratt W.K. Introduction to Digital Image Processing. CRC Press; Boca Raton, FL, USA: 2013.

Zaitoun N.M., Aqel M.J. Survey on image segmentation techniques. Procedia Comput. Sci. 2015;65:797–806. doi: 10.1016/j.procs.2015.09.027. DOI

O’Mahony N., Campbell S., Carvalho A., Harapanahalli S., Hernandez G.V., Krpalkova L., Riordan D., Walsh J. Deep learning vs. traditional computer vision; Proceedings of the Advances in Computer Vision: Proceedings of the 2019 Computer Vision Conference (CVC); Las Vegas, NV, USA. 2–3 May 2019; pp. 128–144.

He K., Zhang X., Ren S., Sun J. Deep residual learning for image recognition; Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition; Las Vegas, NV, USA. 27–30 June 2016; pp. 770–778.

Szegedy C., Ioffe S., Vanhoucke V., Alemi A. Inception-v4, inception-resnet and the impact of residual connections on learning; Proceedings of the AAAI Conference on Artificial Intelligence; San Francisco, CA, USA. 4–9 February 2017.

Lateef F., Ruichek Y. Survey on semantic segmentation using deep learning techniques. Neurocomputing. 2019;338:321–348. doi: 10.1016/j.neucom.2019.02.003. DOI

Ronneberger O., Fischer P., Brox T. U-net: Convolutional networks for biomedical image segmentation; Proceedings of the Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference; Munich, Germany. 5–9 October 2015; pp. 234–241.

Hesamian M.H., Jia W., He X., Kennedy P. Deep learning techniques for medical image segmentation: Achievements and challenges. J. Digit. Imaging. 2019;32:582–596. doi: 10.1007/s10278-019-00227-x. PubMed DOI PMC

Loshchilov I., Hutter F. Fixing Weight Decay Regularization in Adam. 2017. [(accessed on 20 April 2023)]. Available online: https://openreview.net/forum?id=rk6qdGgCZ.

Smith L.N., Topin N. Super-convergence: Very fast training of neural networks using large learning rates; Proceedings of the Artificial Intelligence and Machine Learning for Multi-Domain Operations Applications; Baltimore, MA, USA. 15–17 April 2019; pp. 369–386.

Szeliski R. Handbook of Mathematical Models in Computer Vision. Springer; New York, NY, USA: 2006. Image alignment and stitching; pp. 273–292.

Sotiras A., Davatzikos C., Paragios N. Deformable medical image registration: A survey. IEEE Trans. Med. Imaging. 2013;32:1153–1190. doi: 10.1109/TMI.2013.2265603. PubMed DOI PMC

Song G., Han J., Zhao Y., Wang Z., Du H. A review on medical image registration as an optimization problem. Curr. Med. Imaging. 2017;13:274–283. doi: 10.2174/1573405612666160920123955. PubMed DOI PMC

Maintz J.A., Viergever M.A. A survey of medical image registration. Med. Image Anal. 1998;2:1–36. doi: 10.1016/S1361-8415(01)80026-8. PubMed DOI

Adel E., Elmogy M., Elbakry H. Image stitching based on feature extraction techniques: A survey. Int. J. Comput. Appl. 2014;99:1–8. doi: 10.5120/17374-7818. DOI

Zitova B., Flusser J. Image registration methods: A survey. Image Vis. Comput. 2003;21:977–1000. doi: 10.1016/S0262-8856(03)00137-9. DOI

Thevenaz P., Ruttimann U.E., Unser M. A pyramid approach to subpixel registration based on intensity. IEEE Trans. Image Process. 1998;7:27–41. doi: 10.1109/83.650848. PubMed DOI

Chen X., Diaz-Pinto A., Ravikumar N., Frangi A.F. Deep learning in medical image registration. Prog. Biomed. Eng. 2021;3:012003. doi: 10.1088/2516-1091/abd37c. DOI

Kuang D., Schmah T. Faim–a convnet method for unsupervised 3d medical image registration; Proceedings of the Machine Learning in Medical Imaging: 10th International Workshop, MLMI 2019, Held in Conjunction with MICCAI 2019; Shenzhen, China. 13 October 2019; pp. 646–654.

Jaderberg M., Simonyan K., Zisserman A. Spatial transformer networks. Adv. Neural Inf. Process. Syst. 2015;28 doi: 10.48550/arXiv.1506.02025. DOI

Yan P., Xu S., Rastinehad A.R., Wood B.J. Adversarial image registration with application for MR and TRUS image fusion; Proceedings of the Machine Learning in Medical Imaging: 9th International Workshop, MLMI 2018, Held in Conjunction with MICCAI 2018; Granada, Spain. 16 September 2018; pp. 197–204.

Goodfellow I., Pouget-Abadie J., Mirza M., Xu B., Warde-Farley D., Ozair S., Courville A., Bengio Y. Generative adversarial networks. Commun. ACM. 2020;63:139–144. doi: 10.1145/3422622. DOI

Dalca A., Rakic M., Guttag J., Sabuncu M. Learning conditional deformable templates with convolutional networks. Adv. Neural Inf. Process. Syst. 2019:32. doi: 10.48550/arXiv.1908.02738. DOI

Steffensen J. Interpolation. Courier Corporation; North Chelmsford, MA, USA: 2006.

Yonghong J. Digital Image Processing. 2nd ed. Prentice Hall Press; Hoboken, NJ, USA: 2010.

Wang S., Yang K. An image scaling algorithm based on bilinear interpolation with VC++ Tech. Autom. Appl. 2008;27:44–45.

Feng J.-F., Han H.-J. Image enlargement based on non-uniform B-spline interpolation algorithm. J. Comput. Appl. 2010;30:82. doi: 10.3724/SP.J.1087.2010.00082. DOI

Han D. Comparison of commonly used image interpolation methods; Proceedings of the Conference of the 2nd International Conference on Computer Science and Electronics Engineering (ICCSEE 2013); Hangzhou, China. 22–23 March 2013; pp. 1556–1559.

Saltar G., Aiyer A., Meneveau C. Developing Notebook-based Flow Visualization and Analysis Modules for Computational Fluid Dynamics; Proceedings of the APS Division of Fluid Dynamics Meeting Abstracts; Seattle, WA, USA. 23–26 November 2019; p. NP05.021.

Kikinis R., Pieper S.D., Vosburgh K.G. Intraoperative Imaging and Image-Guided Therapy. Springer; Berlin/Heidelberg, Germany: 2013. 3D Slicer: A platform for subject-specific image analysis, visualization, and clinical support; pp. 277–289.

Saha P.K., Borgefors G., di Baja G.S. A survey on skeletonization algorithms and their applications. Pattern Recognit. Lett. 2016;76:3–12. doi: 10.1016/j.patrec.2015.04.006. DOI

Blum H., Nagel R.N. Shape description using weighted symmetric axis features. Pattern Recognit. 1978;10:167–180. doi: 10.1016/0031-3203(78)90025-0. DOI

Lee T.-C., Kashyap R.L., Chu C.-N. Building skeleton models via 3-D medial surface axis thinning algorithms. CVGIP Graph. Model. Image Process. 1994;56:462–478. doi: 10.1006/cgip.1994.1042. DOI

Van der Walt S., Schönberger J.L., Nunez-Iglesias J., Boulogne F., Warner J.D., Yager N., Gouillart E., Yu T. scikit-image: Image processing in Python. PeerJ. 2014;2:e453. doi: 10.7717/peerj.453. PubMed DOI PMC

Nunez-Iglesias J., Blanch A.J., Looker O., Dixon M.W., Tilley L. A new Python library to analyse skeleton images confirms malaria parasite remodelling of the red blood cell membrane skeleton. PeerJ. 2018;6:e4312. doi: 10.7717/peerj.4312. PubMed DOI PMC

Nilsson J., Akenine-Möller T. Understanding ssim. arXiv. 20202006.13846

Dice L.R. Measures of the amount of ecologic association between species. Ecology. 1945;26:297–302. doi: 10.2307/1932409. DOI

Sorensen T.A. A method of establishing groups of equal amplitude in plant sociology based on similarity of species content and its application to analyses of the vegetation on Danish commons. Biol. Skar. 1948;5:1–34.

Ahmed T., Goyal A. StatPearls [Internet] StatPearls Publishing; Tampa, FL, USA: 2022. Endomyocardial biopsy. PubMed

AlJaroudi W.A., Desai M.Y., Tang W.W., Phelan D., Cerqueira M.D., Jaber W.A. Role of imaging in the diagnosis and management of patients with cardiac amyloidosis: State of the art review and focus on emerging nuclear techniques. J. Nucl. Cardiol. 2014;21:271–283. doi: 10.1007/s12350-013-9800-5. PubMed DOI

Lal S., Li A., Allen D., Allen P.D., Bannon P., Cartmill T., Cooke R., Farnsworth A., Keogh A., Dos Remedios C. Best practice biobanking of human heart tissue. Biophys. Rev. 2015;7:399–406. doi: 10.1007/s12551-015-0182-6. PubMed DOI PMC

Mahapatra D., Antony B., Sedai S., Garnavi R. Deformable medical image registration using generative adversarial networks; Proceedings of the 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018); Washington, DC, USA. 4–7 April 2018; pp. 1449–1453.

Arimura H., Soufi M., Kamezawa H., Ninomiya K., Yamada M. Radiomics with artificial intelligence for precision medicine in radiation therapy. J. Radiat. Res. 2019;60:150–157. doi: 10.1093/jrr/rry077. PubMed DOI PMC

Lee J.Y., Lee K.-s., Seo B.K., Cho K.R., Woo O.H., Song S.E., Kim E.-K., Lee H.Y., Kim J.S., Cha J. Radiomic machine learning for predicting prognostic biomarkers and molecular subtypes of breast cancer using tumor heterogeneity and angiogenesis properties on MRI. Eur. Radiol. 2022;32:650–660. doi: 10.1007/s00330-021-08146-8. PubMed DOI

Peng W.K. Clustering Nuclear Magnetic Resonance: Machine learning assistive rapid two-dimensional relaxometry mapping. Eng. Rep. 2021;3:e12383. doi: 10.1002/eng2.12383. DOI

Kickingereder P., Götz M., Muschelli J., Wick A., Neuberger U., Shinohara R.T., Sill M., Nowosielski M., Schlemmer H.P., Radbruch A., et al. Large-scale Radiomic Profiling of Recurrent Glioblastoma Identifies an Imaging Predictor for Stratifying Anti-Angiogenic Treatment Response. Clin. Cancer Res. 2016;22:5765–5771. doi: 10.1158/1078-0432.CCR-16-0702. PubMed DOI PMC