BACKGROUND: Several diagnostic prediction models to help clinicians discriminate between benign and malignant adnexal masses are available. This study is a head-to-head comparison of the performance of the Assessment of Different NEoplasias in the adneXa (ADNEX) model with that of the Risk of Ovarian Malignancy Algorithm (ROMA). METHODS: This is a retrospective study based on prospectively included consecutive women with an adnexal tumour scheduled for surgery at five oncology centres and one non-oncology centre in four countries between 2015 and 2019. The reference standard was histology. Model performance for ADNEX and ROMA was evaluated regarding discrimination, calibration, and clinical utility. RESULTS: The primary analysis included 894 patients, of whom 434 (49%) had a malignant tumour. The area under the receiver operating characteristic curve (AUC) was 0.92 (95% CI 0.88-0.95) for ADNEX with CA125, 0.90 (0.84-0.94) for ADNEX without CA125, and 0.85 (0.80-0.89) for ROMA. ROMA, and to a lesser extent ADNEX, underestimated the risk of malignancy. Clinical utility was highest for ADNEX. ROMA had no clinical utility at decision thresholds <27%. CONCLUSIONS: ADNEX had better ability to discriminate between benign and malignant adnexal tumours and higher clinical utility than ROMA. CLINICAL TRIAL REGISTRATION: clinicaltrials.gov NCT01698632 and NCT02847832.

Department of Biomedical Data Sciences Leiden University Medical Centre Leiden The Netherlands

Department of Clinical Sciences Malmö Lund University Lund Sweden

Department of Development and Regeneration KU Leuven Leuven Belgium

Department of Gynecologic Oncology National Cancer Institute of Milan Milan Italy

Department of Laboratory Medicine UZ Leuven Leuven Belgium

Department of Obstetrics and Gynaecology 1st Faculty of Medicine Charles University Prague Czech Republic

Department of Obstetrics and Gynaecology University Hospitals Leuven Leuven Belgium

Department of Obstetrics and Gynecology Skåne University Hospital Malmö Sweden

Department of Obstetrics and Gynecology Ziekenhuis Oost Limburg Genk Belgium

Department of Oncology Gynaecological Oncology KU Leuven Leuven Cancer Institute Leuven Belgium

Department of Oncology Laboratory of Tumour Immunology and Immunotherapy Leuven Cancer Institute KU Leuven Leuven Belgium

Department of Pathology UZ Leuven Leuven Belgium

Dipartimento Scienze della Salute della Donna del Bambino e di Sanità Pubblica Fondazione Policlinico Universitario Agostino Gemelli IRCCS Rome Italy

General University Hospital Prague Czech Republic

Leuven Unit for Health Technology Assessment Research KU Leuven Leuven Belgium

Oncoinvent AS Oslo Norway

Queen Charlotte's and Chelsea Hospital Imperial College London UK

Zobrazit více v PubMed

Engelen MJA, Kos HE, Willemse PHB, Aalders JG, de Vries EGE, Schaapveld M, et al. Surgery by consultant gynecologic oncologists improves survival in patients with ovarian carcinoma. Cancer. 2006;106:589–98. doi: 10.1002/cncr.21616. PubMed DOI

Earle CC, Schrag D, Neville BA, Yabroff KR, Topor M, Fahey A, et al. Effect of surgeon specialty on processes of care and outcomes for ovarian cancer patients. J Natl Cancer Inst. 2006;98:172–80. doi: 10.1093/jnci/djj019. PubMed DOI

Woo YL, Kyrgiou M, Bryant A, Everett T, Dickinson HO. Centralisation of services for gynaecological cancers—a Cochrane systematic review. Gynecol Oncol. 2012;126:286–90. doi: 10.1016/j.ygyno.2012.04.012. PubMed DOI

Bristow RE, Chang J, Ziogas A, Anton-Culver H. Adherence to treatment guidelines for ovarian cancer as a measure of quality care. Obstet Gynecol. 2013;121:1226–34. doi: 10.1097/AOG.0b013e3182922a17. PubMed DOI

Jacobs I, Oram D, Fairbanks J, Turner J, Frost C, Grudzinskas JG. A risk of malignancy index incorporating CA 125, ultrasound and menopausal status for the accurate preoperative diagnosis of ovarian cancer. Br J Obstet Gynaecol. 1990;97:922–9. doi: 10.1111/j.1471-0528.1990.tb02448.x. PubMed DOI

Geomini P, Kruitwagen R, Bremer GL, Cnossen J, Mol BWJ. The accuracy of risk scores in predicting ovarian malignancy: a systematic review. Obst Gynecol. 2009;113:384–94. doi: 10.1097/AOG.0b013e318195ad17. PubMed DOI

Kaijser J, Sayasneh A, Van Hoorde K, Ghaem-Maghami S, Bourne T, Timmerman D, et al. Presurgical diagnosis of adnexal tumours using mathematical models and scoring systems: a systematic review and meta-analysis. Hum Reprod Update. 2014;20:449–62. doi: 10.1093/humupd/dmt059. PubMed DOI

Meys EMJ, Kaijser J, Kruitwagen RFPM, Slangen BFM, Van Calster B, Aertgeerts B, et al. Subjective assessment versus ultrasound models to diagnose ovarian cancer: a systematic review and meta-analysis. Eur J Cancer. 2016;58:17–29. doi: 10.1016/j.ejca.2016.01.007. PubMed DOI

Ortiz-Muñoz B, Aznar-Oroval E, García AG, Peris AC, Ballestero PP, Yepes MS, et al. HE4, Ca125 and ROMA algorithm for differential diagnosis between benign gynaecological diseases and ovarian cancer. Tumor Biol. 2014;35:7249–58. doi: 10.1007/s13277-014-1945-6. PubMed DOI

Moore RG, McMeekin DS, Brown AK, DiSilvestro P, Miller MC, Allard WJ, et al. A novel multiple marker bioassay utilizing HE4 and CA125 for the prediction of ovarian cancer in patients with a pelvic mass. Gynecol Oncol. 2009;112:40–6. doi: 10.1016/j.ygyno.2008.08.031. PubMed DOI PMC

Huang J, Chen J, Huang Q. Diagnostic value of HE4 in ovarian cancer: a meta-analysis. Eur J Obstet Gynecol Reprod Biol. 2018;231:35–42. doi: 10.1016/j.ejogrb.2018.10.008. PubMed DOI

Van Calster B, Van Hoorde K, Valentin L, Testa AC, Fischerova D, Van Holsbeke C, et al. Evaluating the risk of ovarian cancer before surgery using the ADNEX model to differentiate between benign, borderline, early and advanced stage invasive, and secondary metastatic tumours: prospective multicentre diagnostic study. BMJ. 2014;349:g5920. doi: 10.1136/bmj.g5920. PubMed DOI PMC

Westwood M, Ramaekers B, Lang S, Grimm S, Deshpande S, de Kock S, et al. Risk scores to guide referral decisions for people with suspected ovarian cancer in secondary care: a systematic review and cost-effectiveness analysis. Health Technol Assess. 2018;22:1–264. doi: 10.3310/hta22440. PubMed DOI PMC

Timmerman D, Planchamp F, Bourne T, Landolfo C, du Bois A, Chiva L, et al. ESGO/ISUOG/IOTA/ESGE consensus statement on pre-operative diagnosis of ovarian tumors. Int J Gynecol Cancer. 2021;31:961–82. doi: 10.1136/ijgc-2021-002565. PubMed DOI PMC

Collins GS, Moons KGM. Comparing risk prediction models. BMJ. 2012;344:e3186. doi: 10.1136/bmj.e3186. PubMed DOI

Qian L, Du Q, Jiang M, Yuan F, Chen H, Feng W. Comparison of the diagnostic performances of ultrasound-based models for predicting malignancy in patients with adnexal masses. Front Oncol. 2021;11:673722. doi: 10.3389/fonc.2021.673722. PubMed DOI PMC

Meys EMJ, Jeelof LS, Achten NMJ, Slangen BFM, Lambrechts S, Kruitwagen RFPM, et al. Estimating risk of malignancy in adnexal masses: external validation of the ADNEX model and comparison with other frequently used ultrasound methods. Ultrasound Obstet Gynecol. 2017;49:784–92. doi: 10.1002/uog.17225. PubMed DOI PMC

Stukan M, Badocha M, Ratajczak K. Development and validation of a model that includes two ultrasound parameters and the plasma D-dimer level for predicting malignancy in adnexal masses: an observational study. BMC Cancer. 2019;19:564. doi: 10.1186/s12885-019-5629-x. PubMed DOI PMC

Van Calster B, Valentin L, Froyman W, Landolfo C, Ceusters J, Testa AC, et al. Validation of models to diagnose ovarian cancer in patients managed surgically or conservatively: multicentre cohort study. BMJ. 2020;370:m2614. doi: 10.1136/bmj.m2614. PubMed DOI PMC

Czekierdowski A, Stachowicz N, Smolen A, Łoziński T, Guzik P, Kluz T. Performance of IOTA simple rules risks, ADNEX model, subjective assessment compared to CA125 and HE4 with ROMA algorithm in discriminating between benign, borderline and stage I malignant adnexal lesions. Diagnostics. 2023;13:885. doi: 10.3390/diagnostics13050885. PubMed DOI PMC

Collins GS, Reitsma JB, Altman DG, Moons KGM. Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): the TRIPOD statement. BMJ. 2015;350:g7594. doi: 10.1136/bmj.g7594. PubMed DOI

Timmerman D, Valentin L, Bourne TH, Collins WP, Verrelst H, Vergote I. Terms, definitions and measurements to describe the sonographic features of adnexal tumors: a consensus opinion from the International Ovarian Tumor Analysis (IOTA) group. Ultrasound Obstet Gynecol. 2000;16:500–5. doi: 10.1046/j.1469-0705.2000.00287.x. PubMed DOI

Installé AJ, Van den Bosch T, De Moor B, Timmerman D. Clinical data miner: an electronic case report form system with integrated data preprocessing and machine-learning libraries supporting clinical diagnostic model research. JMIR Med Inform. 2014;2:e28. doi: 10.2196/medinform.3251. PubMed DOI PMC

Kurman RJ, Carcangiu M, Herrington CS. World Health Organisation classification of tumours of the female reproductive organs, 4th ed. Lyon: International Agency for Research on Cancer; 2014.

Prat J. FIGO Committee on Gynecologic Oncology. Staging classification for cancer of the ovary, fallopian tube, and peritoneum. Int J Gynecol Obstet. 2014;124:1–5. doi: 10.1016/j.ijgo.2013.10.001. PubMed DOI

Timmerman D, Testa AC, Bourne T, Ferrazzi E, Ameye L, Konstantinovic ML, et al. Logistic regression model to distinguish between the benign and malignant adnexal mass before surgery: a multicenter study by the International Ovarian Tumor Analysis Group. J Clin Oncol. 2005;23:8794–801. doi: 10.1200/JCO.2005.01.7632. PubMed DOI

Debray TP, Damen JA, Riley RD, Snell K, Reitsma JB, Hooft L, et al. A framework for meta-analysis of prediction model studies with binary and time-to-event outcomes. Stat Methods Med Res. 2019;28:2768–86. doi: 10.1177/0962280218785504. PubMed DOI PMC

Demler OV, Pencina MJ, D’Agostino RB. Misuse of DeLong test to compare AUCs for nested models. Stat Med. 2012;31:2577–87. doi: 10.1002/sim.5328. PubMed DOI PMC

Van Calster B, Nieboer D, Vergouwe Y, De Cock B, Pencina MJ, Steyerberg EW. A calibration hierarchy for risk models was defined: from utopia to empirical data. J Clin Epidemiol. 2016;74:167–76. doi: 10.1016/j.jclinepi.2015.12.005. PubMed DOI

Wynants L, Vergouwe Y, Van Huffel S, Timmerman D, Van Calster B. Does ignoring clustering in multicenter data influence the performance of prediction models? A simulation study. Stat Methods Med Res. 2018;27:1723–36. doi: 10.1177/0962280216668555. PubMed DOI

Reitsma JB, Glas AS, Rutjes AWS, Scholten RJPM, Bossuyt PM, Zwinderman AH. Bivariate analysis of sensitivity and specificity produces informative summary measures in diagnostic reviews. J Clin Epidemiol. 2005;58:982–90. doi: 10.1016/j.jclinepi.2005.02.022. PubMed DOI

Wynants L, Timmerman D, Verbakel JY, Testa A, Savelli L, Fischerova D, et al. Clinical utility of risk models to refer patients with adnexal masses to specialized oncology care: multicenter external validation using decision curve analysis. Clin Cancer Res. 2017;23:5082–90. doi: 10.1158/1078-0432.CCR-16-3248. PubMed DOI

Vickers AJ, Van Calster B, Steyerberg EW. Net benefit approaches to the evaluation of prediction models, molecular markers, and diagnostic tests. BMJ. 2016;352:i6. 10.1136/bmj.i6. PubMed PMC

Wynants L, Riley RD, Timmerman D, Van Calster B. Random-effects meta-analysis of the clinical utility of tests and prediction models. Stat Med. 2018;37:2034–52. doi: 10.1002/sim.7653. PubMed DOI

Van Calster B, Vergouwe Y, Looman CWN, Van Belle V, Timmerman D, Steyerberg EW. Assessing the discriminative ability of risk models for more than two outcome categories. Eur J Epidemiol. 2012;27:761–70. doi: 10.1007/s10654-012-9733-3. PubMed DOI

van Buuren S, Groothuis-Oudshoorn K. mice: multivariate imputation by chained equations in R. J Stat Softw. 2011;45:1–67. doi: 10.18637/jss.v045.i03. DOI

Vickers AJ, van Calster B, Steyerberg EW. A simple, step-by-step guide to interpreting decision curve analysis. Diagn Progn Res. 2019;3:18. doi: 10.1186/s41512-019-0064-7. PubMed DOI PMC

Tso E, Elson P, VanLente F, Markman M. The “real-life” variability of CA-125 in ovarian cancer patients. Gynecol Oncol. 2006;103:141–4. doi: 10.1016/j.ygyno.2006.02.010. PubMed DOI

Sayasneh A, Ferrara L, De Cock B, Saso S, Al-Memar M, Johnson S, et al. Evaluating the risk of ovarian cancer before surgery using the ADNEX model: a multicentre external validation study. Br J Cancer. 2016;115:542–8. doi: 10.1038/bjc.2016.227. PubMed DOI PMC

Van Calster B, McLernon DJ, van Smeden M, Wynants L, Steyerberg EW. Calibration: the Achilles heel of predictive analytics. BMC Med. 2019;17:230. doi: 10.1186/s12916-019-1466-7. PubMed DOI PMC

Van Calster B, Steyerberg EW, Wynants L, van Smeden M. There is no such thing as a validated prediction model. BMC Med. 2023;21:70. doi: 10.1186/s12916-023-02779-w. PubMed DOI PMC

Youssef A, Pencina M, Thakur A, Zhu T, Clifton D, Shah NH. External validation of AI models in health should be replaced with recurring local validation. Nat Med. 2023;29:2686–7. doi: 10.1038/s41591-023-02540-z. PubMed DOI

Tuxen MK, Sölétormos G, Petersen PH, Schiøler V, Dombernowsky P. Assessment of biological variation and analytical imprecision of CA 125, CEA, and TPA in relation to monitoring of ovarian cancer. Gynecol Oncol. 1999;74:12–22. doi: 10.1006/gyno.1999.5414. PubMed DOI

Braga F, Ferraro S, Mozzi R, Panteghini M. The importance of individual biology in the clinical use of serum biomarkers for ovarian cancer. Clin Chem Lab Med. 2014;52:1625–31. doi: 10.1515/cclm-2014-0097. PubMed DOI

Ferraro S, Borille S, Carnevale A, Frusciante E, Bassani N, Panteghini M. Verification of the harmonization of human epididymis protein 4 assays. Clin Chem Lab Med. 2016;54:1635–43. doi: 10.1515/cclm-2015-1142. PubMed DOI

Barr CE, Funston G, Mounce LTA, Pemberton PW, Howe JD, Crosbie EJ. Comparison of two immunoassays for the measurement of serum HE4 for ovarian cancer. Pract Lab Med. 2021;26:e00235. doi: 10.1016/j.plabm.2021.e00235. PubMed DOI PMC

Engelen MJA, de Bruijn HWA, Hollema H, ten Hoor KA, Willemse PHB, Aalders JG, et al. Serum CA 125, carcinoembryonic antigen, and CA 19-9 as tumor markers in borderline ovarian tumors. Gynecol Oncol. 2000;78:16–20. doi: 10.1006/gyno.2000.5811. PubMed DOI

Gotlieb WH, Soriano D, Achiron R, Zalel Y, Davidson B, Kopolovic J, et al. CA 125 measurement and ultrasonography in borderline tumors of the ovary. Am J Obstet Gynecol. 2000;183:541–6. doi: 10.1067/mob.2000.105940. PubMed DOI

Sevinc A, Adli M, Kalender ME, Camci C. Benign causes of increased serum CA-125 concentration. Lancet Oncol. 2007;8:1054–5. doi: 10.1016/S1470-2045(07)70357-1. PubMed DOI

Van Calster B, Timmerman D, Bourne T, Testa AC, Van Holsbeke C, Domali E, et al. Discrimination between benign and malignant adnexal masses by specialist ultrasound examination versus serum CA-125. J Natl Cancer Inst. 2007;99:1706–14. doi: 10.1093/jnci/djm199. PubMed DOI

Huhtinen K, Suvitie P, Hiissa J, Junnila J, Huvila J, Kujari H, et al. Serum HE4 concentration differentiates malignant ovarian tumours from ovarian endometriotic cysts. Br J Cancer. 2009;100:1315–9. doi: 10.1038/sj.bjc.6605011. PubMed DOI PMC

Sokalska A, Timmerman D, Testa AC, Van Holsbeke C, Lissoni AA, Leone FPG, et al. Diagnostic accuracy of transvaginal ultrasound examination for assigning a specific diagnosis to adnexal masses. Ultrasound Obstet Gynecol. 2009;34:462–70. doi: 10.1002/uog.6444. PubMed DOI

Valentin L. Pattern recognition of pelvic masses by gray-scale ultrasound imaging: the contribution of Doppler ultrasound. Ultrasound Obstet Gynecol. 1999;14:338–47. doi: 10.1046/j.1469-0705.1999.14050338.x. PubMed DOI

Sayasneh A, Kaijser J, Preisler J, Smith AA, Raslan F, Johnson S, et al. Accuracy of ultrasonography performed by examiners with varied training and experience in predicting specific pathology of adnexal masses. Ultrasound Obstet Gynecol. 2015;45:605–12. doi: 10.1002/uog.14675. PubMed DOI

Zobrazit více v PubMed

ClinicalTrials.gov
NCT02847832, NCT01698632