Circulating tumor DNA (ctDNA) sequencing is being rapidly adopted in precision oncology, but the accuracy, sensitivity and reproducibility of ctDNA assays is poorly understood. Here we report the findings of a multi-site, cross-platform evaluation of the analytical performance of five industry-leading ctDNA assays. We evaluated each stage of the ctDNA sequencing workflow with simulations, synthetic DNA spike-in experiments and proficiency testing on standardized, cell-line-derived reference samples. Above 0.5% variant allele frequency, ctDNA mutations were detected with high sensitivity, precision and reproducibility by all five assays, whereas, below this limit, detection became unreliable and varied widely between assays, especially when input material was limited. Missed mutations (false negatives) were more common than erroneous candidates (false positives), indicating that the reliable sampling of rare ctDNA fragments is the key challenge for ctDNA assays. This comprehensive evaluation of the analytical performance of ctDNA assays serves to inform best practice guidelines and provides a resource for precision oncology.

Accugenomics Inc Wilmington NC USA

Agilent Technologies Cedar Creek TX USA

Agilent Technologies La Jolla CA USA

Agilent Technologies Santa Clara CA USA

Astrazeneca Pharmaceuticals Waltham MA USA

Australian Institute of Bioengineering and Nanotechnology University of Queensland Queensland QLD Australia

Bioinformatics and Computational Biology Laboratory National Heart Lung and Blood Institute National Institutes of Health Bethesda MD USA

Bioinformatics Integrated DNA Technologies Inc Coralville IA USA

Cancer Genetics Inc Rutherford NJ USA

Cancer Theme Garvan Institute of Medical Research Sydney NSW Australia

Center for Bioinformatics and Computational Biology and the Institute of Biomedical Sciences School of Life Sciences East China Normal University Shanghai China

Center of Molecular Medicine Central European Institute of Technology Masaryk University Brno Czech Republic

Clinical Diagnostic Division Thermo Fisher Scientific Fremont CA USA

Clinical Laboratory Burning Rock Biotech Guangzhou China

Clinical Sequencing Division Thermo Fisher Scientific Austin TX USA

Clinical Sequencing Division Thermo Fisher Scientific South San Francisco CA USA

CMINDS Research Center Department of Electrical and Computer Engineering College of Engineering University of Massachusetts Lowell Lowell MA USA

Department of Genetics University of North Carolina Chapel Hill NC USA

Department of Immunology Genomics and Microarray Core Facility University of Texas Southwestern Medical Center Dallas TX USA

Department of Information Science University of Arkansas at Little Rock Little Rock AR USA

Department of Physiology and Biophysics Weill Cornell Medicine Cornell University New York NY USA

Departments of Medicine Pathology and Cancer Biology College of Medicine and Life Sciences University of Toledo Health Sciences Campus Toledo OH USA

Departments of Pathology and Pediatrics University of Utah School of Medicine Salt Lake City UT USA

Division of Bioinformatics and Biostatistics National Center for Toxicological Research US Food and Drug Administration Jefferson AR USA

EATRIS ERIC European Infrastructure for Translational Medicine Amsterdam The Netherlands

Elim Biopharmaceuticals Inc Hayward CA USA

Fudan Gospel Joint Research Center for Precision Medicine Fudan University Shanghai China

Genomics and Epigenetics Theme Garvan Institute of Medical Research Sydney NSW Australia

Human Phenome Institute Fudan University Shanghai China

Icahn Institute and Department of Genetics and Genomic Sciences Icahn School of Medicine at Mount Sinai New York NY USA

Illumina Inc San Diego CA USA

Immuneering Corporation Cambridge MA USA

Institute for Molecular Medicine Finland University of Helsinki Helsinki Finland

Institute for Personalized Cancer Therapy MD Anderson Cancer Center Houston TX USA

Intramural Research Program Laboratory of Epidemiology and Population Sciences National Institute on Aging National Institutes of Health Baltimore MD USA

Kinghorn Centre for Clinical Genomics Garvan Institute of Medical Research Sydney NSW Australia

Market and Application Development Bioinformatics Roche Sequencing Solutions Inc Pleasanton CA USA

Marketing Integrated DNA Technologies Inc Coralville IA USA

Massachusetts General Hospital Harvard Medical School Boston MA USA

National Institute of Environmental Health Sciences Research Triangle Park Morrisville NC USA

NGS Products and Services Integrated DNA Technologies Inc Coralville IA USA

NIHR Biomedical Research Centre Royal Marsden Hospital Sutton Surrey UK

OmniSeq Inc Buffalo NY USA

Primbio Genes Biotechnology East Lake High tech Development Zone Wuhan Hubei China

Q2 Solutions EA Genomics Morrisville NC USA

R and D Genomics MPS Institute for Clinical and Experimental Pathology ARUP Laboratories Salt Lake City UT USA

Research and Development Burning Rock Biotech Shanghai China

Research and Development Integrated DNA Technologies Inc Coralville IA USA

Research and Development QIAGEN Sciences Inc Frederick MD USA

ResearchDx Inc Irvine CA USA

Roche Sequencing Solutions Inc Pleasanton CA USA

St Vincent's Clinical School Faculty of Medicine University of New South Wales Sydney NSW Australia

St Vincent's Clinical School University of New South Wales Sydney NSW Australia

Stanford Genome Technology Center Stanford University Palo Alto CA USA

State Key Laboratory of Genetic Engineering School of Life Sciences and Shanghai Cancer Hospital Cancer Institute Fudan University Shanghai China

Winthrop P Rockefeller Cancer Institute University of Arkansas for Medical Sciences Little Rock AR USA

Zobrazit více v PubMed

Leon SA, Shapiro B, Sklaroff DM & Yaros MJ Free DNA in the serum of cancer patients and the effect of therapy. Cancer Res 37, 646–650 (1977). PubMed

Stroun M, Anker P, Lyautey J, Lederrey C & Maurice PA Isolation and characterization of DNA from the plasma of cancer patients. Eur J Cancer Clin Oncol 23, 707–712 (1987). PubMed

Abbosh C et al.Phylogenetic ctDNA analysis depicts early-stage lung cancer evolution. Nature 545, 446–451 (2017). PubMed PMC

Bettegowda C et al.Detection of circulating tumor DNA in early- and late-stage human malignancies. Sci Transl Med 6, 224ra24 (2014). PubMed PMC

Abbosh C, Birkbak NJ & Swanton C Early stage NSCLC - challenges to implementing ctDNA-based screening and MRD detection. Nat Rev Clin Oncol 15, 577–586 (2018). PubMed

Aggarwal C et al.Strategies for the successful implementation of plasma-based NSCLC genotyping in clinical practice. Nat Rev Clin Oncol (2020). PubMed

Aravanis AM, Lee M & Klausner RD Next-Generation Sequencing of Circulating Tumor DNA for Early Cancer Detection. Cell 168, 571–574 (2017). PubMed

Siravegna G, Marsoni S, Siena S & Bardelli A Integrating liquid biopsies into the management of cancer. Nat Rev Clin Oncol 14, 531–548 (2017). PubMed

Wan JCM et al.Liquid biopsies come of age: towards implementation of circulating tumour DNA. Nat Rev Cancer 17, 223–238 (2017). PubMed

Volckmar A-L et al.A field guide for cancer diagnostics using cell-free DNA: From principles to practice and clinical applications. Genes Chromosomes Cancer 57, 123–139 (2017). PubMed

Ross MG et al.Characterizing and measuring bias in sequence data. Genome Biol. 14, R51 (2013). PubMed PMC

Brannon AR et al.Enhanced specificity of high sensitivity somatic variant profiling in cell-free DNA via paired normal sequencing: design, validation, and clinical experience of the MSK-ACCESS liquid biopsy assay. bioRxiv 2020.06.27.175471 (2020).

Clark TA et al.Analytical Validation of a Hybrid Capture–Based Next-Generation Sequencing Clinical Assay for Genomic Profiling of Cell-Free Circulating Tumor DNA. J Mol Diagn 20, 686–702 (2018). PubMed PMC

Dawson S-J et al.Analysis of circulating tumor DNA to monitor metastatic breast cancer. N Engl J Med 368, 1199–1209 (2013). PubMed

Forshew T et al.Noninvasive identification and monitoring of cancer mutations by targeted deep sequencing of plasma DNA. Sci Transl Med 4, 136ra68 (2012). PubMed

Kinde I, Wu J, Papadopoulos N, Kinzler KW & Vogelstein B Detection and quantification of rare mutations with massively parallel sequencing. Proc Natl Acad Sci U S A 108, 9530–9535 (2011). PubMed PMC

Klein EA et al.Development of a comprehensive cell-free DNA (cfDNA) assay for early detection of multiple tumor types: The Circulating Cell-free Genome Atlas (CCGA) study. J Clin Oncol 36, 12021–12021 (2018).

Miller AM et al.Tracking tumour evolution in glioma through liquid biopsies of cerebrospinal fluid. Nature 565, 654–658 (2019). PubMed PMC

Murtaza M et al.Non-invasive analysis of acquired resistance to cancer therapy by sequencing of plasma DNA. Nature 497, 108–112 (2013). PubMed

Murtaza M et al.Multifocal clonal evolution characterized using circulating tumour DNA in a case of metastatic breast cancer. Nat Commun 6, 8760 (2015). PubMed PMC

Newman AM et al.An ultrasensitive method for quantitating circulating tumor DNA with broad patient coverage. Nat Med 20, 548–554 (2014). PubMed PMC

Newman AM et al.Integrated digital error suppression for improved detection of circulating tumor DNA. Nat Biotechnol 34, 547–555 (2016). PubMed PMC

Odegaard JI et al.Validation of a Plasma-Based Comprehensive Cancer Genotyping Assay Utilizing Orthogonal Tissue- and Plasma-Based Methodologies. Clin Cancer Res 24, 3539 (2018). PubMed

Plagnol V et al.Analytical validation of a next generation sequencing liquid biopsy assay for high sensitivity broad molecular profiling. PLoS One 13, e0193802 (2018). PubMed PMC

Kuderer NM et al.Comparison of 2 commercially available next-generation sequencing platforms in oncology. JAMA Oncol 3, 996–998 (2017). PubMed PMC

Stetson D et al.Orthogonal comparison of four plasma NGS tests with tumor suggests technical factors are a major source of assay discordance. JCO Precision Oncology 1–9 (2019). PubMed

Torga G & Pienta KJ Patient-paired sample congruence between 2 commercial liquid biopsy tests. JAMA Oncol 4, 868–870 (2018). PubMed PMC

Merker JD et al.Circulating tumor DNA analysis in patients with cancer: American Society of Clinical Oncology and College of American Pathologists Joint Review. J Clin Oncol 36, 1631–1641 (2018). PubMed

Shiraishi Y et al.A comprehensive characterization of cis-acting splicing-associated variants in human cancer. Genome Res. 28, 1111–1125 (2018). PubMed PMC

Bos JL The ras gene family and human carcinogenesis. Mutat Res 195, 255–271 (1988). PubMed

Blackburn J et al.Use of synthetic DNA spike-in controls (sequins) for human genome sequencing. Nat Protoc 14, 2119–2151 (2019). PubMed

Deveson IW et al.Chiral DNA sequences as commutable controls for clinical genomics. Nat Commun 1–13 (2019). PubMed PMC

Horn S et al.TERT promoter mutations in familial and sporadic melanoma. Science 339, 959–961 (2013). PubMed

Jones W et al.A Verified Genomic Reference Sample for Assessing Performance of Cancer Panels Detecting Small Variants of Low Allele Frequency. Genome Biology (2021). 10.1186/s13059-021-02316-z. PubMed DOI PMC

Fu GK, Hu J, Wang P-H & Fodor SPA Counting individual DNA molecules by the stochastic attachment of diverse labels. Proc Natl Acad Sci U S A 108, 9026–9031 (2011). PubMed PMC

Saito T & Rehmsmeier M The precision-recall plot is more informative than the ROC plot when evaluating binary classifiers on imbalanced datasets. PLoS One 10, e0118432 (2015). PubMed PMC

Hardwick SA, Deveson IW & Mercer TR Reference standards for next-generation sequencing. Nat Rev Genet 18, 473–484 (2017). PubMed

Hodis E et al.A landscape of driver mutations in melanoma. Cell 150, 251–263 (2012). PubMed PMC

Sheridan C Investors keep the faith in cancer liquid biopsies. Nat Biotechnol 37, 972–974 (2019). PubMed

Lanman RB et al.Analytical and clinical validation of a digital sequencing panel for quantitative, highly accurate evaluation of cell-free circulating tumor DNA. PLoS One 10, e0140712 (2015). PubMed PMC

Phallen J et al.Direct detection of early-stage cancers using circulating tumor DNA. Sci Transl Med 9, (2017). PubMed PMC

Cohen JD et al.Detection and localization of surgically resectable cancers with a multi-analyte blood test. Science 359, 926–930 (2018). PubMed PMC

Tie J et al.Circulating tumor DNA analysis detects minimal residual disease and predicts recurrence in patients with stage II colon cancer. Sci Transl Med 8, 346ra92 (2016). PubMed PMC

Diehl F et al.Circulating mutant DNA to assess tumor dynamics. Nat Med 14, 985–990 (2008). PubMed PMC

Mouliere F et al.Enhanced detection of circulating tumor DNA by fragment size analysis. Sci Transl Med 10, (2018). PubMed PMC

Underhill HR et al.Fragment Length of Circulating Tumor DNA. PLoS Genet 12, e1006162 (2016). PubMed PMC

Shen SY et al.Sensitive tumour detection and classification using plasma cell-free DNA methylomes. Nature 563, 579–583 (2018). PubMed

Kim Y-W et al.Monitoring circulating tumor DNA by analyzing personalized cancer-specific rearrangements to detect recurrence in gastric cancer. Experimental & Molecular Medicine 51, 1–10 (2019). PubMed PMC

Klega K et al.Detection of somatic structural variants enables quantification and characterization of circulating tumor DNA in children with solid tumors. JCO Precision Oncology 2018, 10.1200/PO.17.00285 (2018). PubMed DOI PMC

Peng H et al.CNV detection from circulating tumor DNA in late stage non-small cell lung cancer patients. Genes (Basel) 10, 926 (2019). PubMed PMC

Cai Z et al.Detection of microsatellite instability from circulating tumor DNA by targeted deep sequencing. J Mol Diagn 22, 860–870 (2020). PubMed

Hu Y et al.False-positive plasma genotyping due to clonal hematopoiesis. Clin Cancer Res 24, 4437–4443 (2018). PubMed

McKenna A et al.The Genome Analysis Toolkit: a MapReduce framework for analyzing next-generation DNA sequencing data. Genome Res 20, 1297–1303 (2010). PubMed PMC

Li H & Durbin R Fast and accurate short read alignment with Burrows–Wheeler transform. Bioinformatics 25, 1754–1760 (2009). PubMed PMC

Koboldt DC et al.VarScan 2: somatic mutation and copy number alteration discovery in cancer by exome sequencing. Genome Res 22, 568–576 (2012). PubMed PMC

A verified genomic reference sample for assessing performance of cancer panels detecting small variants of low allele frequency