OBJECTIVES: To assess the ability, as well as factors affecting the ability, of ultrasound examiners with different levels of ultrasound experience to detect correctly infiltration of ovarian cancer in predefined anatomical locations, and to evaluate the inter-rater agreement regarding the presence or absence of cancer infiltration, using preacquired ultrasound videoclips obtained in a selected patient sample with a high prevalence of cancer spread. METHODS: This study forms part of the Imaging Study in Advanced ovArian Cancer multicenter observational study (NCT03808792). Ultrasound videoclips showing assessment of infiltration of ovarian cancer were obtained by the principal investigator (an ultrasound expert, who did not participate in rating) at 19 predefined anatomical sites in the abdomen and pelvis, including five sites that, if infiltrated, would indicate tumor non-resectability. For each site, there were 10 videoclips showing cancer infiltration and 10 showing no cancer infiltration. The reference standard was either findings at surgery with histological confirmation or response to chemotherapy. For statistical analysis, the 19 sites were grouped into four anatomical regions: pelvis, middle abdomen, upper abdomen and lymph nodes. The videoclips were assessed by raters comprising both senior gynecologists (mainly self-trained expert ultrasound examiners who perform preoperative ultrasound assessment of ovarian cancer spread almost daily) and gynecologists who had undergone a minimum of 6 months' supervised training in the preoperative ultrasound assessment of ovarian cancer spread in a gynecological oncology center. The raters were classified as highly experienced or less experienced based on annual individual caseload and the number of years that they had been performing ultrasound evaluation of ovarian cancer spread. Raters were aware that for each site there would be 10 videoclips with and 10 without cancer infiltration. Each rater independently classified every videoclip as showing or not showing cancer infiltration and rated the image quality (on a scale from 0 to 10) and their diagnostic confidence (on a scale from 0 to 10). A generalized linear mixed model with random effects was used to estimate which factors (including level of experience, image quality, diagnostic confidence and anatomical region) affected the likelihood of a correct classification of cancer infiltration. We assessed the observed percentage of videoclips classified correctly, the expected percentage of videoclips classified correctly based on the generalized linear mixed model and inter-rater agreement (reliability) in classifying anatomical sites as being infiltrated by cancer. RESULTS: Twenty-five raters participated in the study, of whom 13 were highly experienced and 12 were less experienced. The observed percentage of correct classification of cancer infiltration ranged from 70% to 100% depending on rater and anatomical site, and the median percentage of correct classification for the 25 raters ranged from 90% to 100%. The probability of correct classification of all 380 videoclips ranged from 0.956 to 0.975 and was not affected by the rater's level of ultrasound experience. The likelihood of correct classification increased with increased image quality and diagnostic confidence and was affected by anatomical region. It was highest for sites in the pelvis, second highest for those in the middle abdomen, third highest for lymph nodes and lowest for sites in the upper abdomen. The inter-rater agreement of all 25 raters regarding the presence of cancer infiltration ranged from substantial (Fleiss kappa, 0.68 (95% CI, 0.66-0.71)) to very good (Fleiss kappa, 0.99 (95% CI, 0.97-1.00)) depending on the anatomical site. It was lowest for sites in the upper abdomen (Fleiss kappa, 0.68 (95% CI, 0.66-0.71) to 0.97 (95% CI, 0.94-0.99)) and highest for sites in the pelvis (Fleiss kappa, 0.94 (95% CI, 0.92-0.97) to 0.99 (95% CI, 0.97-1.00)). CONCLUSIONS: Ultrasound examiners with different levels of ultrasound experience can classify correctly predefined anatomical sites as being infiltrated or not infiltrated by ovarian cancer based on video recordings obtained by an experienced ultrasound examiner, and the inter-rater agreement is substantial. The likelihood of correct classification as well as the inter-rater agreement is highest for sites in the pelvis and lowest for sites in the upper abdomen. However, owing to the study design, our results regarding diagnostic accuracy and inter-rater agreement are likely to be overoptimistic. © 2025 The Author(s). Ultrasound in Obstetrics & Gynecology published by John Wiley & Sons Ltd on behalf of International Society of Ultrasound in Obstetrics and Gynecology.

1st Faculty of Medicine Charles University and General University Hospital Prague Prague Czech Republic

Department of Clinical Sciences Malmö Lund University Lund Sweden

Department of Computer Science Faculty of Electrical Engineering Czech Technical University Prague Prague Czech Republic

Department of Gynecologic Oncology Fondazione IRCCS Istituto Nazionale dei Tumori Milan Italy

Department of Gynecology Portuguese Institute of Oncology Francisco Gentil Lisbon Portugal

Department of Obstetrics and Gynecology Clinica Universidad de Navarra University of Navarra Pamplona Spain

Department of Obstetrics and Gynecology Skåne University Hospital Malmö Sweden

Faculty of Mathematics and Physics Charles University Prague Czech Republic

Gynecologic Oncology Centre Department of Gynecology Obstetrics and Neonatology 1st Faculty of Medicine Charles University and General University Hospital Prague Prague Czech Republic

Gynecologic Oncology Unit Department of Woman and Child Health and Public Health Fondazione Policlinico Universitario A Gemelli IRCCS Rome Italy

Masaryk University Institute of Biostatistics and Analyses Brno Czech Republic

Preventive Gynecology Unit Division of Gynecology European Institute of Oncology IRCCS Milan Italy

QuironSalud Hospital Málaga Spain

Section of Obstetrics and Gynecology University Department of Life Sciences and Public Health Università Cattolica del Sacro Cuore Rome Italy

Zobrazit více v PubMed

Ledermann JA, Matias‐Guiu X, Amant F, et al. ESGO‐ESMO‐ESP consensus conference recommendations on ovarian cancer: pathology and molecular biology and early, advanced and recurrent disease. Ann Oncol. 2024;35:248‐266. PubMed

Fischerova D, Pinto P, Burgetova A, et al. Preoperative staging of ovarian cancer: comparison between ultrasound, CT and whole‐body diffusion‐weighted MRI (ISAAC study). Ultrasound Obstet Gynecol. 2022;59:248‐262. PubMed

Moruzzi MC, Bolomini G, Esposito R, et al. Diagnostic performance of ultrasound in assessing the extension of disease in advanced ovarian cancer. Am J Obstet Gynecol. 2022;227:601.e1‐601.e20. PubMed

Alcázar JL, Caparros M, Arraiza M, et al. Pre‐operative assessment of intra‐abdominal disease spread in epithelial ovarian cancer: a comparative study between ultrasound and computed tomography. Int J Gynecol Cancer. 2019;29:227‐233. PubMed

Fischerova D, Smet C, Scovazzi U, Sousa DN, Hundarova K, Haldorsen IS. Staging by imaging in gynecologic cancer and the role of ultrasound: an update of European joint consensus statements. Int J Gynecol Cancer. 2024;34:363‐378. PubMed PMC

Kottner J, Audigé L, Brorson S, et al. Guidelines for Reporting Reliability and Agreement Studies (GRRAS) were proposed. J Clin Epidemiol. 2011;64:96‐106. PubMed

Colombo N, Sessa C, Bois AD, et al. ESMO‐ESGO consensus conference recommendations on ovarian cancer: pathology and molecular biology, early and advanced stages, borderline tumours and recurrent disease. Int J Gynecol Cancer. 2019;29:728‐760. PubMed

Education and Practical Standards Committee , European Federation of Societies for Ultrasound in Medicine and Biology . Minimum training recommendations for the practice of medical ultrasound. Ultraschall Med. 2006;27:79‐105. PubMed

Fleiss JL. Measuring nominal scale agreement among many raters. Psychol Bull. 1971;5:378‐382.

Brennan P, Silman A. Statistical methods for assessing observer variability in clinical measures. BMJ. 1992;304:1491‐1494. PubMed PMC

Prat J. Staging classification for cancer of the ovary, fallopian tube, and peritoneum. Int J Gynaecol Obstet. 2014;124:1‐5. PubMed

Fischerova D, Zikan M, Semeradova I, et al. Ultrasound in preoperative assessment of pelvic and abdominal spread in patients with ovarian cancer: a prospective study. Ultrasound Obstet Gynecol. 2017;49:263‐274. PubMed

Luis Alcázar J, Ramón Pérez‐Vidal J, Tameish S, Chacón E, Manzour N, Ángel MJ. Ultrasound for assessing tumor spread in ovarian cancer. A systematic review of the literature and meta‐analysis. Eur J Obstet Gynecol Reprod Biol. 2024;292:194‐200. PubMed

Boussuges A, Rives S, Finance J, Brégeon F. Assessment of diaphragmatic function by ultrasonography: current approach and perspectives. World J Clin Cases. 2020;8:2408‐2424. PubMed PMC

Pálsdóttir K, Fridsten S, Blomqvist L, et al. Interobserver agreement of transvaginal ultrasound and magnetic resonance imaging in local staging of cervical cancer. Ultrasound Obstet Gynecol. 2021;58:773‐779. PubMed PMC

Eriksson LS, Lindqvist PG, Floter Radestad A, et al. Transvaginal ultrasound assessment of myometrial and cervical stromal invasion in women with endometrial cancer: interobserver reproducibility among ultrasound experts and gynecologists. Ultrasound Obstet Gynecol. 2015;45:476‐482. PubMed

Martins WP, Nastri CO. Interpreting reproducibility results for ultrasound measurements. Ultrasound Obstet Gynecol. 2014;43:479‐480. PubMed

Coelho Neto MA, Roncato P, Nastri CO, Martins WP. True Reproducibility of UltraSound Techniques (TRUST): systematic review of reliability studies in obstetrics and gynecology. Ultrasound Obstet Gynecol. 2015;46:14‐20. PubMed

Zobrazit více v PubMed

ClinicalTrials.gov
NCT03808792