PURPOSE OF REVIEW: Pathomics, the fusion of digitalized pathology and artificial intelligence, is currently changing the landscape of medical pathology and biologic disease classification. In this review, we give an overview of Pathomics and summarize its most relevant applications in urology. RECENT FINDINGS: There is a steady rise in the number of studies employing Pathomics, and especially deep learning, in urology. In prostate cancer, several algorithms have been developed for the automatic differentiation between benign and malignant lesions and to differentiate Gleason scores. Furthermore, several applications have been developed for the automatic cancer cell detection in urine and for tumor assessment in renal cancer. Despite the explosion in research, Pathomics is not fully ready yet for widespread clinical application. SUMMARY: In prostate cancer and other urologic pathologies, Pathomics is avidly being researched with commercial applications on the close horizon. Pathomics is set to improve the accuracy, speed, reliability, cost-effectiveness and generalizability of pathology, especially in uro-oncology.

Department of Pathology Sorbonne University Assistance Publique Hôpitaux de Paris Hopital Tenon Paris France

Department of Urology 2nd Faculty of Medicine Charles University Prague Czech Republic

Department of Urology Comprehensive Cancer Center Vienna General Hospital Medical University of Vienna Vienna Austria

Department of Urology University Hospital of Tours Tours

Department of Urology University Medical Center Hamburg Eppendorf Hamburg Germany

Department of Urology University of Texas Southwestern Dallas Texas USA

Department of Urology Weill Cornell Medical College New York New York

European Association of Urology Research Foundation Arnhem Netherlands

Institute for Urology and Reproductive Health Sechenov University Moscow Russia

Karl Landsteiner Institute of Urology and Andrology Vienna Austria

See more in PubMed

Gupta R, Kurc T, Sharma A, et al. The emergence of Pathomics. Curr Pathobiol Rep 2019; 10.1007/s40139-019-00200-x. DOI

Louis DN, Feldman M, Carter AB, et al. Computational pathology: a path ahead. Arch Pathol Lab Med 2016; 140:41–50.

Liu Y, Chen PC, Krause J, Peng L. How to read articles that use machine learning: users’ guides to the medical literature. JAMA 2019; 322:1806–1816.

Faes L, Liu X, Wagner S, et al. A clinician's guide to artificial intelligence: how to critically appraise machine learning studies. Transl Vis Sci Technol 2020; 9:7.

Anwar SM, Majid M, Qayyum A, et al. Medical image analysis using convolutional neural networks: a review. J Med Syst 2018; 42:226.

Munir K, Elahi H, Ayub A, et al. Cancer diagnosis using deep learning: a bibliographic review. Cancers (Basel) 2019; 11:1235.

LeCun Y, Bengio Y, Hinton G. Deep learning. Nature 2015; 521:436–444.

Saltz J, Almeida J, Gao Y, et al. Towards generation, management, and exploration of combined radiomics and pathomics datasets for cancer research. AMIA Jt Summits Transl Sci Proc 2017; 2017:85–94.

Banna GL, Olivier T, Rundo F, et al. The promise of digital biopsy for the prediction of tumor molecular features and clinical outcomes associated with immunotherapy. Front Med (Lausanne) 2019; 6:172.

Xu J, Janowczyk A, Chandran S, Madabhushi A. A high-throughput active contour scheme for segmentation of histopathological imagery. Med Image Anal 2011; 15:851–862.

Lee G, Sparks R, Ali S, et al. Co-occurring gland angularity in localized subgraphs: predicting biochemical recurrence in intermediate-risk prostate cancer patients. PLoS One 2014; 9:e97954.

Alvi E, Gupta R, Borok RZ, et al. Overview of established and emerging immunohistochemical biomarkers and their role in correlative studies in MRI. J Magn Reson Imaging 2020; 51:341–354.

Serag A, Ion-Margineanu A, Qureshi H, et al. Translational AI and deep learning in diagnostic pathology. Front Med 2019; 6:185.

Ehteshami Bejnordi B, Veta M, Johannes van Diest P, et al. Diagnostic assessment of deep learning algorithms for detection of lymph node metastases in women with breast cancer. JAMA 2017; 318:2199–2210.

Deo RC. Machine learning in medicine. Circulation 2015; 132:1920–1930.

Madabhushi A, Lee G. Image analysis and machine learning in digital pathology: challenges and opportunities. Med Image Anal 2016; 33:170–175.

Suarez-Ibarrola R, Hein S, Reis G, et al. Current and future applications of machine and deep learning in urology: a review of the literature on urolithiasis, renal cell carcinoma, and bladder and prostate cancer. World J Urol 2019.

Nam S, Chong Y, Jung CK, et al. Introduction to digital pathology and computer-aided pathology. J Pathol Transl Med 2020; 54:125–134.

Wang X, Yang W, Weinreb J, et al. Searching for prostate cancer by fully automated magnetic resonance imaging classification: deep learning versus nondeep learning. Sci Rep 2017; 7:15415.

Mnih V, Kavukcuoglu K, Silver D, et al. Human-level control through deep reinforcement learning. Nature 2015; 518:529–533.

Kweldam CF, Nieboer D, Algaba F, et al. Gleason grade 4 prostate adenocarcinoma patterns: an interobserver agreement study among genitourinary pathologists. Histopathology 2016; 69:441–449.

Allsbrook WC Jr, Mangold KA, Johnson MH, et al. Interobserver reproducibility of Gleason grading of prostatic carcinoma: general pathologist. Hum Pathol 2001; 32:81–88.

Rizzardi AE, Zhang X, Vogel RI, et al. Quantitative comparison and reproducibility of pathologist scoring and digital image analysis of estrogen receptor beta2 immunohistochemistry in prostate cancer. Diagn Pathol 2016; 11:63.

Ström P, Kartasalo K, Olsson H, et al. Artificial intelligence for diagnosis and grading of prostate cancer in biopsies: a population-based, diagnostic study. Lancet Oncol 2020; 21:222–232.

Monaco JP, Tomaszewski JE, Feldman MD, et al. High-throughput detection of prostate cancer in histological sections using probabilistic pairwise Markov models. Med Image Anal 2010; 14:617–629.

Doyle S, Feldman M, Tomaszewski J, Madabhushi A. A boosted Bayesian multiresolution classifier for prostate cancer detection from digitized needle biopsies. IEEE Trans Biomed Eng 2012; 59:1205–1218.

Gorelick L, Veksler O, Gaed M, et al. Prostate histopathology: learning tissue component histograms for cancer detection and classification. IEEE Trans Med Imaging 2013; 32:1804–1818.

Nguyen K, Sarkar A, Jain AK. Prostate cancer grading: use of graph cut and spatial arrangement of nuclei. IEEE Trans Med Imaging 2014; 33:2254–2270.

Lucas M, Jansen I, Savci-Heijink CD, et al. Deep learning for automatic Gleason pattern classification for grade group determination of prostate biopsies. Virchows Arch 2019; 475:77–83.

Gertych A, Ing N, Ma Z, et al. Machine learning approaches to analyze histological images of tissues from radical prostatectomies. Comput Med Imaging Graph 2015; 46 (Pt 2):197–208.

Mosquera-Lopez C, Agaian S, Velez-Hoyos A, Thompson I. Computer-aided prostate cancer diagnosis from digitized histopathology: a review on texture-based systems. IEEE Rev Biomed Eng 2015; 8:98–113.

Nir G, Karimi D, Goldenberg SL, et al. Comparison of artificial intelligence techniques to evaluate performance of a classifier for automatic grading of prostate cancer from digitized histopathologic images. JAMA Netw Open 2019; 2:e190442.

Avenel C, Tolf A, Dragomir A, Carlbom IB. Glandular segmentation of prostate cancer: an illustration of how the choice of histopathological stain is one key to success for computational pathology. Front Bioeng Biotechnol 2019; 7:125.

Singh M, Kalaw EM, Giron DM, et al. Gland segmentation in prostate histopathological images. J Med Imaging (Bellingham) 2017; 4:027501.

Tennill TA, Gross ME, Frieboes HB. Automated analysis of co-localized protein expression in histologic sections of prostate cancer. PLoS One 2017; 12:e0178362.

Blom S, Paavolainen L, Bychkov D, et al. Systems pathology by multiplexed immunohistochemistry and whole-slide digital image analysis. Sci Rep 2017; 7:15580.

Litjens G, Sánchez CI, Timofeeva N, et al. Deep learning as a tool for increased accuracy and efficiency of histopathological diagnosis. Sci Rep 2016; 6:26286.

Källén H, Molin J, Heyden A, et al. Towards grading gleason score using generically trained deep convolutional neural networks. 2016 IEEE 13th International Symposium on Biomedical Imaging (ISBI); 2016 13-16 April 2016.

Zhou N, Fedorov A, Fennessy F, et al. Large scale digital prostate pathology image analysis combining feature extraction and deep neural network. 2017.

Jiménez del Toro O, Atzori M, Otálora S, et al. Convolutional neural networks for an automatic classification of prostate tissue slides with high-grade Gleason score: SPIE; 2017.

Ren J, Sadimin E, Foran DJ, Qi X. Computer aided analysis of prostate histopathology images to support a refined Gleason grading system. Proc SPIE Int Soc Opt Eng 2017; 10133.

Kwak JT, Hewitt S. Lumen-based detection of prostate cancer via convolutional neural networks: SPIE; 2017.

Kwak JT, Hewitt S. Nuclear architecture analysis of prostate cancer via convolutional neural networks. IEEE Access 2017; 5:18526–18533.

Li J, Sarma KV, Chung Ho K, et al. A multiscale U-Net for semantic segmentation of histological images from radical prostatectomies. AMIA Annu Symp Proc 2017; 2017:1140–1148.

Gummeson A, Arvidsson I, Ohlsson M, et al. Automatic Gleason grading of H and E stained microscopic prostate images using deep convolutional neural networks: SPIE; 2017.

Ma Z, Li J, Salemi H, et al. Semantic segmentation for prostate cancer grading by convolutional neural networks 2018. 46 p.

Nir G, Hor S, Karimi D, et al. Automatic grading of prostate cancer in digitized histopathology images: learning from multiple experts. Med Image Anal 2018; 50:167–180.

Arvaniti E, Fricker KS, Moret M, et al. Automated Gleason grading of prostate cancer tissue microarrays via deep learning. Sci Rep 2018; 8:12054.

Singh M, Kalaw E, Wang J, et al. Cribriform pattern detection in prostate histopathological images using deep learning models 2019.

Kott O, Linsley D, Amin A, et al. Development of a deep learning algorithm for the histopathologic diagnosis and Gleason grading of prostate cancer biopsies: a pilot study. Eur Urol Focus 2019.

Karimi D, Nir G, Fazli L, et al. Deep learning-based Gleason grading of prostate cancer from histopathology images: role of multiscale decision aggregation and data augmentation. IEEE J Biomed Health Inform 2019; 5:1413–1426.

Nagpal K, Foote D, Liu Y, et al. Development and validation of a deep learning algorithm for improving Gleason scoring of prostate cancer. NPJ Digit Med 2019; 2:48.

Bulten W, Bándi P, Hoven J, et al. Epithelium segmentation using deep learning in H&E-stained prostate specimens with immunohistochemistry as reference standard. Sci Rep 2019; 9:864.

Poojitha UP, Lal Sharma S. Hybrid unified deep learning network for highly precise Gleason grading of prostate cancer. Conf Proc IEEE Eng Med Biol Soc 2019; 2019:899–903.

Otalora S, Atzori M, Andrearczyk V, et al. Staining invariant features for improving generalization of deep convolutional neural networks in computational pathology. Front Bioeng Biotechnol 2019; 7:198.

Chen CM, Huang YS, Fang PW, et al. A computer-aided diagnosis system for differentiation and delineation of malignant regions on whole-slide prostate histopathology image using spatial statistics and multidimensional DenseNet. Med Phys 2020; 47:1021–1033.

Bulten W, Pinckaers H, van Boven H, et al. Automated deep-learning system for Gleason grading of prostate cancer using biopsies: a diagnostic study. Lancet Oncol 2020; 21:233–241.

Donovan MJ, Fernandez G, Scott R, et al. Development and validation of a novel automated Gleason grade and molecular profile that define a highly predictive prostate cancer progression algorithm-based test. Prostate Cancer Prostatic Dis 2018; 21:594–603.

Bhargava HK, Leo P, Elliott R, et al. Computationally derived image signature of stromal morphology is prognostic of prostate cancer recurrence following prostatectomy in African American patients. Clin Cancer Res 2020; 26:1915–1923.

Ing N, Tomczak J, Miller E, et al., editors. A deep multiple instance model to predict prostate cancer metastasis from nuclear morphology. Conference on Medical Imaging with Deep Learning; 2018.

Ersvaer E, Hveem TS, Vlatkovic L, et al. Prognostic value of DNA ploidy and automated assessment of stroma fraction in prostate cancer. Int J Cancer 2019; 147:1228–1234.

Lee G, Veltri RW, Zhu G, et al. Nuclear shape and architecture in benign fields predict biochemical recurrence in prostate cancer patients following radical prostatectomy: preliminary findings. Eur Urol Focus 2017; 3:457–466.

Harder N, Athelogou M, Hessel H, et al. Tissue Phenomics for prognostic biomarker discovery in low- and intermediate-risk prostate cancer. Sci Rep 2018; 8:4470.

Schaumberg AJ, Rubin MA, Fuchs TJ, amp H. H&E-stained whole slide image deep learning predicts SPOP mutation state in prostate cancer. bioRxiv 2018; 064279.

Liu D, Takhar M, Alshalalfa M, et al. Impact of the SPOP mutant subtype on the interpretation of clinical parameters in prostate cancer. JCO Precision Oncol 2018; 2018:1–13.

Zhang YD, Wang J, Wu CJ, et al. An imaging-based approach predicts clinical outcomes in prostate cancer through a novel support vector machine classification. Oncotarget 2016; 7:78140–78151.

Takeuchi T, Hattori-Kato M, Okuno Y, et al. Prediction of prostate cancer by deep learning with multilayer artificial neural network. Can Urol Assoc J 2019; 13:E145–E150.

McGarry SD, Hurrell SL, Iczkowski KA, et al. Radio-pathomic maps of epithelium and lumen density predict the location of high-grade prostate cancer. Int J Radiat Oncol Biol Phys 2018; 101:1179–1187.

Shao CH, Chen CL, Lin JY, et al. Metabolite marker discovery for the detection of bladder cancer by comparative metabolomics. Oncotarget 2017; 8:38802–38810.

Sokolov I, Dokukin ME, Kalaparthi V, et al. Noninvasive diagnostic imaging using machine-learning analysis of nanoresolution images of cell surfaces: detection of bladder cancer. Proc Natl Acad Sci U S A 2018; 115:12920–12925.

Poropatich K, Yang JC, Goyal R, et al. Nuclear size measurement for distinguishing urothelial carcinomas from reactive urothelium on tissue sections. Diagn Pathol 2016; 11:57.

Borkowski T, Monika Dulewicz A, Borkowski A, et al. Clinical value of digital image analysis in the diagnosis of urinary bladder cancer, particularly in aggressive tumors: a preliminary report. Pol J Pathol 2016; 67:122–129.

Harmon SA, Sanford TH, Brown GT, et al. Multiresolution application of artificial intelligence in digital pathology for prediction of positive lymph nodes from primary tumors in bladder cancer. JCO Clin Cancer Inform 2020; 4:367–382.

Brieu N, Gavriel CG, Nearchou IP, et al. Automated tumour budding quantification by machine learning augments TNM staging in muscle-invasive bladder cancer prognosis. Sci Rep 2019; 9:5174.

McConkey DJ, Choi W. Molecular subtypes of bladder cancer. Curr Oncol Rep 2018; 20:77.

Seiler R, Ashab HAD, Erho N, et al. Impact of molecular subtypes in muscle-invasive bladder cancer on predicting response and survival after neoadjuvant chemotherapy. Eur Urol 2017; 72:544–554.

Tan TZ, Rouanne M, Tan KT, et al. Molecular subtypes of urothelial bladder cancer: results from a meta-cohort analysis of 2411 tumors. Eur Urol 2019; 75:423–432.

Woerl AC, Eckstein M, Geiger J, et al. Deep learning predicts molecular subtype of muscle-invasive bladder cancer from conventional histopathological slides. Eur Urol 2020; 78:256–264.

Tian K, Rubadue C, Lin D, et al. Automated clear cell renal carcinoma grade classification with prognostic significance. PLOS ONE 2019; 14:e0222641.

Holdbrook DA, Singh M, Choudhury Y, et al. Automated renal cancer grading using nuclear pleomorphic patterns. JCO Clin Cancer Inform 2018; 2:1–12.

Baghdadi A, Aldhaam NA, Elsayed AS, et al. Automated differentiation of benign renal oncocytoma and chromophobe renal cell carcinoma on computed tomography using deep learning. BJU Int 2020; 125:553–560.

Tabibu S, Vinod PK, Jawahar CV. Pan-renal cell carcinoma classification and survival prediction from histopathology images using deep learning. Sci Rep 2019; 9:10509.

Linder N, Taylor JC, Colling R, et al. Deep learning for detecting tumour-infiltrating lymphocytes in testicular germ cell tumours. J Clin Pathol 2019; 72:157–164.

Tsehay Y, Lay N, Wang X, et al., editors. Biopsy-guided learning with deep convolutional neural networks for prostate cancer detection on multiparametric MRI. 2017 IEEE 14th International Symposium on Biomedical Imaging (ISBI 2017); 2017 18-21 April 2017.

Faes L, Wagner S, Fu D, et al. Automated deep learning design for medical image classification by health-care professionals with no coding experience: a feasibility study. Lancet Digital Health 2019; 1:e232–e242.

Bensalah K, Montorsi F, Shariat SF. Challenges of cancer biomarker profiling. Eur Urol 2007; 52:1601–1609.

Shariat SF, Lotan Y, Vickers A, et al. Statistical consideration for clinical biomarker research in bladder cancer. Urol Oncol 2010; 28:389–400.

Acs B, Rantalainen M, Hartman J. Artificial intelligence as the next step towards precision pathology. J Intern Med 2020; 288:62–81.

A novel assessment of whole-mount Gleason grading in prostate cancer to identify candidates for radical prostatectomy: a machine learning-based multiomics study