OBJECTIVE: ANCA-associated vasculitis (AAV) is a relapsing-remitting disease, resulting in incremental tissue injury. The gold-standard relapse definition (Birmingham Vasculitis Activity Score, BVAS>0) is often missing or inaccurate in registry settings, leading to errors in ascertainment of this key outcome. We sought to create a computable phenotype (CP) to automate retrospective identification of relapse using real-world data in the research setting. METHODS: We studied 536 patients with AAV and >6 months follow-up recruited to the Rare Kidney Disease registry (a national longitudinal, multicentre cohort study). We followed five steps: (1) independent encounter adjudication using primary medical records to assign the ground truth, (2) selection of data elements (DEs), (3) CP development using multilevel regression modelling, (4) internal validation and (5) development of additional models to handle missingness. Cut-points were determined by maximising the F1-score. We developed a web application for CP implementation, which outputs an individualised probability of relapse. RESULTS: Development and validation datasets comprised 1209 and 377 encounters, respectively. After classifying encounters with diagnostic histopathology as relapse, we identified five key DEs; DE1: change in ANCA level, DE2: suggestive blood/urine tests, DE3: suggestive imaging, DE4: immunosuppression status, DE5: immunosuppression change. F1-score, sensitivity and specificity were 0.85 (95% CI 0.77 to 0.92), 0.89 (95% CI 0.80 to 0.99) and 0.96 (95% CI 0.93 to 0.99), respectively. Where DE5 was missing, DE2 plus either DE1/DE3 were required to match the accuracy of BVAS. CONCLUSIONS: This CP accurately quantifies the individualised probability of relapse in AAV retrospectively, using objective, readily accessible registry data. This framework could be leveraged for other outcomes and relapsing diseases.

1st Faculty of Medicine Charles University Prague Czech Republic

ADAPT SFI centre Trinity College Dublin Dublin Ireland

Department of Computer Science and Statistics Trinity College Dublin Dublin Ireland

Department of Immunology St James's Hospital Dublin Ireland

Department of Mathematical Science University of Durham Durham UK

Department of Nephrology Cork University Hospital Cork Ireland

Department of Nephrology General University Hospital Prague Czech Republic

Department of Statistics Dublin Institute of Technology Dublin Ireland

Division of Rheumatology Department of Medicine Division of Epidemiology Department of Biostatistics Epidemiology and Informatics University of Pennsylvania Philadelphia Pennsylvania USA

General University Hospital Prague Czech Republic

National Centre for Pharmacoeconomics St James's Hospital Dublin Ireland

Nuffield Department of Orthopaedics Rheumatology and Musculoskeletal Science University of Oxford Oxford UK

School of Computer Science and Statistics Trinity College Dublin Dublin Ireland

School of Infection and Immunity University of Glasgow Glasgow UK

School of Medicine College of Medicine Nursing and Health Science University of Galway Galway Ireland

Trinity Kidney Centre Trinity Translational Medicine Institute Trinity College Dublin Dublin Ireland

Vasculitis Ireland Awareness Dublin Ireland

Zobrazit více v PubMed

Stone JH, Merkel PA, Spiera R, et al. . Rituximab versus cyclophosphamide for ANCA-associated vasculitis. N Engl J Med 2010;363:221–32. 10.1056/NEJMoa0909905 PubMed DOI PMC

Smith RM, Jones RB, Specks U, et al. . Rituximab as therapy to induce remission after relapse in ANCA-associated vasculitis. Ann Rheum Dis 2020;79:1243–9. 10.1136/annrheumdis-2019-216863 PubMed DOI PMC

Robson J, Doll H, Suppiah R, et al. . Damage in the ANCA-associated Vasculitides: long-term data from the European vasculitis study group (EUVAS) therapeutic trials. Ann Rheum Dis 2015;74:177–84. 10.1136/annrheumdis-2013-203927 PubMed DOI

Salama AD. Relapse in anti-neutrophil cytoplasm antibody (ANCA)–Associated vasculitis. Kidney Int Rep 2020;5:7–12. 10.1016/j.ekir.2019.10.005 PubMed DOI PMC

Wester Trejo MAC, Floßmann O, Westman KW, et al. . Renal relapse in antineutrophil cytoplasmic autoantibody-associated vasculitis: unpredictable, but predictive of renal outcome. Rheumatology 2019;58:103–9. 10.1093/rheumatology/key260 PubMed DOI

Little MA, Nightingale P, Verburgh CA, et al. . Early mortality in systemic vasculitis: relative contribution of adverse events and active vasculitis. Ann Rheum Dis 2010;69:1036–43. 10.1136/ard.2009.109389 PubMed DOI

Mukhtyar C, Lee R, Brown D, et al. . Modification and validation of the Birmingham vasculitis activity score (version 3). Ann Rheum Dis 2009;68:1827–32. 10.1136/ard.2008.101279 PubMed DOI

Berti A, Specks U. Remission maintenance in ANCA-associated vasculitis: does one size fit all Expert Rev Clin Immunol 2019;15:1273–86. 10.1080/1744666X.2020.1693260 PubMed DOI

McAdoo SP, Medjeral-Thomas N, Gopaluni S, et al. . Long-term follow-up of a combined Rituximab and cyclophosphamide regimen in renal anti-neutrophil cytoplasm antibody-associated vasculitis. Nephrol Dial Transplant 2019;34:63–73. 10.1093/ndt/gfx378 PubMed DOI PMC

van Dam LS, Dirikgil E, Bredewold EW, et al. . Proteinase-3-anti-neutrophil cytoplasmic antibodies (Pr3-ANCAs) predict relapses in ANCA–associated vasculitis patients after Rituximab. Nephrol Dial Transplant 2020. 10.1093/ndt/gfaa066 PubMed DOI PMC

Kemna MJ, Damoiseaux J, Austen J, et al. . ANCA as a Predictor of relapse: useful in patients with renal involvement but not in patients with Nonrenal disease. J Am Soc Nephrol 2015;26:537–42. 10.1681/ASN.2013111233 PubMed DOI PMC

King C, Druce KL, Nightingale P, et al. . Predicting relapse in anti-neutrophil cytoplasmic antibody-associated vasculitis: a systematic review and meta-analysis. Rheumatol Adv Pract 2021;5:rkab018. 10.1093/rap/rkab018 PubMed DOI PMC

Richesson RL, Green BB, Laws R, et al. . Pragmatic (trial) Informatics: a perspective from the NIH health care systems research Collaboratory. J Am Med Inform Assoc 2017;24:996–1001. 10.1093/jamia/ocx016 PubMed DOI PMC

Collaboratory NHCSR . Electronic health records-based phenotyping. In: Rethinking clinical trials: a living textbook of pragmatic clinical trials, 27 June 2014. Available: https://sites.duke.edu/rethinkingclinicaltrials/ehr-phenotyping/

Sreih AG, Annapureddy N, Springer J, et al. . Development and validation of case-finding Algorithms for the identification of patients with anti-neutrophil cytoplasmic antibody-associated vasculitis in large Healthcare administrative databases. Pharmacoepidemiol Drug Saf 2016;25:1368–74. 10.1002/pds.4116 PubMed DOI PMC

Scott J, Nic An Ríogh E, Al Nokhatha S, et al. . ANCA-associated vasculitis in Ireland: a multi-centre national cohort study [version 1; peer review: 2 approved]. HRB Open Res 2022;5:80. 10.12688/hrbopenres.13651.1 PubMed DOI PMC

Harris PA, Taylor R, Minor BL, et al. . The Redcap consortium: building an international community of software platform partners. J Biomed Inform 2019;95:103208. 10.1016/j.jbi.2019.103208 PubMed DOI PMC

Harris PA, Taylor R, Thielke R, et al. . Research electronic data capture (Redcap)--A Metadata-driven methodology and Workflow process for providing Translational research Informatics support. J Biomed Inform 2009;42:377–81. 10.1016/j.jbi.2008.08.010 PubMed DOI PMC

Watts R, Lane S, Hanslik T, et al. . Development and validation of a consensus methodology for the classification of the ANCA-associated Vasculitides and Polyarteritis Nodosa for Epidemiological studies. Ann Rheum Dis 2007;66:222–7. 10.1136/ard.2006.054593 PubMed DOI PMC

Jones RB, Tervaert JWC, Hauser T, et al. . Rituximab versus cyclophosphamide in ANCA-associated renal vasculitis. N Engl J Med 2010;363:211–20. 10.1056/NEJMoa0909169 PubMed DOI

Chowdhury MZI, Turin TC. Variable selection strategies and its importance in clinical prediction Modelling. Fam Med Community Health 2020;8:e000262. 10.1136/fmch-2019-000262 PubMed DOI PMC

Fox J, Monette G. Generalized Collinearity diagnostics. J Am Stat Assoc 1992;87:178. 10.2307/2290467 DOI

Bates D, Mächler M, Bolker B, et al. . Fitting linear mixed-effects models Usinglme4. J Stat Softw 2015;67. 10.18637/jss.v067.i01 DOI

Network TE. Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): The TRIPOD statement, 2021. Available: https://www.equator-network.org/reporting-guidelines/tripod-statement\ PubMed PMC

Kirkwood BS. Essential Medical Statistics. 2nd edn. Oxford: Blackwell Science, 2003.

Saito T, Rehmsmeier M. The precision-recall plot is more informative than the ROC plot when evaluating binary classifiers on imbalanced Datasets. PLOS ONE 2015;10:e0118432. 10.1371/journal.pone.0118432 PubMed DOI PMC

Alba AC, Agoritsas T, Walsh M, et al. . Discrimination and calibration of clinical prediction models: users’ guides to the medical literature. JAMA 2017;318:1377–84. 10.1001/jama.2017.12126 PubMed DOI

Pan S, Wu A, Weiner M, et al. . Development and evaluation of Computable phenotypes in pediatric Epilepsy:3 cases. J Child Neurol 2021;36:990–7. 10.1177/08830738211019578 PubMed DOI

Johnson M. Feature engineering and selection: A practical approach for predictive models: Taylor & Francis. 2019. Available: https://www.taylorfrancis.com/books/9781351609470

McGlinn K, Rutherford MA, Gisslander K, et al. . FAIRVASC: A semantic web approach to rare disease Registry integration. Comput Biol Med 2022;145:105313. 10.1016/j.compbiomed.2022.105313 PubMed DOI

Smith R, Jayne D, Merkel PA. Op0026 a randomized, controlled trial of Rituximab versus azathioprine after induction of remission with Rituximab for patients with ANCA-associated vasculitis and relapsing disease. Ann Rheum Dis 2020;79:19–20. 10.1136/annrheumdis-2020-eular.2717 PubMed DOI

Monach PA, Warner RL, Lew R, et al. . Serum biomarkers of disease activity in Longitudinal assessment of patients with ANCA-associated vasculitis. ACR Open Rheumatol 2022;4:168–76. 10.1002/acr2.11366 PubMed DOI PMC

Moran SM, Scott J, Clarkson MR, et al. . The clinical application of urine soluble Cd163 in ANCA-associated vasculitis. J Am Soc Nephrol 2021;32:2920–32. 10.1681/ASN.2021030382 PubMed DOI PMC

Saczynski JS, Andrade SE, Harrold LR, et al. . A systematic review of validated methods for identifying heart failure using administrative data. Pharmacoepidemiol Drug Saf 2012;21 Suppl 1:129–40. 10.1002/pds.2313 PubMed DOI PMC

Tomasson G, Grayson PC, Mahr A, et al. . The value of rise in anti-neutrophil cytoplasmic antibody (ANCA) measurements for predicting relapse among patients with ANCA-associated Vasculitisa meta-analysis. Arthritis Rheum 2009;60:648.

Al-Soudi A, Vegting Y, Klarenbeek PL, et al. . A systematic review and meta-analysis on the value of serial ANCA level evaluation. Front Med (Lausanne) 2022;9:844112. 10.3389/fmed.2022.844112 PubMed DOI PMC

Tomasson G, Grayson PC, Mahr AD, et al. . Value of ANCA measurements during remission to predict a relapse of ANCA-associated vasculitis-a meta-analysis. Rheumatology (Oxford) 2012;51:100–9. 10.1093/rheumatology/ker280 PubMed DOI PMC

Egan AC, Kronbichler A, Neumann I, et al. . The sound of Interconnectivity; the European vasculitis society 2022 report. Kidney Int Rep 2022;7:1745–57. 10.1016/j.ekir.2022.05.018 PubMed DOI PMC

Ehrenstein V, Petersen I, Smeeth L, et al. . Helping everyone do better: a call for validation studies of routinely recorded health data. Clin Epidemiol 2016;8:49–51. 10.2147/CLEP.S104448 PubMed DOI PMC

Health NIo . Use Common Data Elements for More FAIR Research Data, Available: https://cde.nlm.nih.gov/home

Health NIo . Value Set Authority Centre, Available: https://vsac.nlm.nih.gov/

Bartlett JW, Frost C, Mattsson N, et al. . Determining cut-points for Alzheimer’s disease biomarkers: statistical issues, methods and challenges. Biomark Med 2012;6:391–400. 10.2217/bmm.12.49 PubMed DOI

Chubak J, Pocobelli G, Weiss NS. Tradeoffs between accuracy measures for electronic health care data Algorithms. J Clin Epidemiol 2012;65:343–9. 10.1016/j.jclinepi.2011.09.002 PubMed DOI PMC

Office CS. Census of Population 2022 - Preliminary Results, 2022. Available: https://www.cso.ie/en/releasesandpublications/ep/p-cpr/censusofpopulation2022-preliminaryresults/

Watts RA, Hatemi G, Burns JC, et al. . Global epidemiology of vasculitis. Nat Rev Rheumatol 2022;18:22–34. 10.1038/s41584-021-00718-8 PubMed DOI PMC

Okpara C, Edokwe C, Ioannidis G, et al. . The reporting and handling of missing data in longitudinal studies of older adults is suboptimal: a methodological survey of geriatric journals. BMC Med Res Methodol 2022;22:122. 10.1186/s12874-022-01605-w PubMed DOI PMC

Jenkins DA, Martin GP, Sperrin M, et al. . Continual updating and monitoring of clinical prediction models: time for dynamic prediction systems Diagn Progn Res 2021;5:1. 10.1186/s41512-020-00090-3 PubMed DOI PMC

Consortium WWW . Resource Description Framework (RDF) Model and Syntax Specification, 1999. Available: https://www.w3.org/TR/PR-rdf-syntax/Overview.html

Pfaff ER, Bradford R, Clark M. n.d. Cohort identification using semantic web Technologies: Triplestores as engines for complex Computable Phenotyping. Health Informatics. 10.1101/2021.12.02.21267186 DOI