BACKGROUND: Oncopanel genomic testing, which identifies important somatic variants, is increasingly common in medical practice and especially in clinical trials. Currently, there is a paucity of reliable genomic reference samples having a suitably large number of pre-identified variants for properly assessing oncopanel assay analytical quality and performance. The FDA-led Sequencing and Quality Control Phase 2 (SEQC2) consortium analyze ten diverse cancer cell lines individually and their pool, termed Sample A, to develop a reference sample with suitably large numbers of coding positions with known (variant) positives and negatives for properly evaluating oncopanel analytical performance. RESULTS: In reference Sample A, we identify more than 40,000 variants down to 1% allele frequency with more than 25,000 variants having less than 20% allele frequency with 1653 variants in COSMIC-related genes. This is 5-100× more than existing commercially available samples. We also identify an unprecedented number of negative positions in coding regions, allowing statistical rigor in assessing limit-of-detection, sensitivity, and precision. Over 300 loci are randomly selected and independently verified via droplet digital PCR with 100% concordance. Agilent normal reference Sample B can be admixed with Sample A to create new samples with a similar number of known variants at much lower allele frequency than what exists in Sample A natively, including known variants having allele frequency of 0.02%, a range suitable for assessing liquid biopsy panels. CONCLUSION: These new reference samples and their admixtures provide superior capability for performing oncopanel quality control, analytical accuracy, and validation for small to large oncopanels and liquid biopsy assays.

Agilent Technologies 11011 N Torrey Pines Rd La Jolla CA 92037 USA

Agilent Technologies 1834 State Hwy 71 West Cedar Creek TX 78612 USA

Agilent Technologies 5301 Stevens Creek Blvd Santa Clara CA 95051 USA

Bioinformatics and Computational Biology Laboratory National Heart Lung and Blood Institute National Institutes of Health Bethesda MD 20892 USA

Bioinformatics Integrated DNA Technologies Inc 1710 Commercial Park Coralville IA 52241 USA

Bioinformatics Research and Early Development Roche Sequencing Solutions Inc 1301 Shoreway Rd Suite 7 300 Belmont CA 94002 USA

Bioinformatics Research Institute of Molecular Biotechnology Boku University Vienna Vienna Austria

Center for Bioinformatics and Computational Biology and the Institute of Biomedical Sciences School of Life Sciences East China Normal University 500 Dongchuan Rd Shanghai 200241 China

Center of Genome and Personalized Medicine Institute of Cancer Stem Cell Dalian Medical University Dalian Liaoning China

Center of Molecular Medicine Central European Institute of Technology Masaryk University Kamenice 5 625 00 Brno Czech Republic

Clinical Sequencing Division Thermo Fisher Scientific 180 Oyster Point Blvd South San Francisco CA 94080 USA

College of Chemistry Sichuan University Chengdu 610064 Sichuan China

Department of Biotechnology Boku University Vienna Austria

Department of Computer Science Engineering and Physics University of Michigan Flint Flint MI 48502 USA

Department of Epidemiology School of Public Health Fudan University Shanghai China

Department of Immunology Genomics and Microarray Core Facility University of Texas Southwestern Medical Center 5323 Harry Hine Blvd Dallas TX 75390 USA

Department of Medicine College of Medicine and Life Sciences The University of Toledo Toledo OH 43614 USA

Department of Physiology and Biophysics Weill Cornell Medicine Cornell University New York NY 10065 USA

Departments of Medicine Pathology and Cancer Biology College of Medicine and Life Sciences University of Toledo Health Sciences Campus 3000 Arlington Ave Toledo OH 43614 USA

Division of Bioinformatics and Biostatistics National Center for Toxicological Research US Food and Drug Administration Jefferson AR 72079 USA

Division of Genetic and Molecular Toxicology National Center for Toxicological Research US Food and Drug Administration Jefferson AR 72079 USA

EATRIS ERIC European Infrastructure for Translational Medicine De Boelelaan 1118 1081 HZ Amsterdam The Netherlands

Fondazione Bruno Kessler 38123 Trento Italy

Fudan Gospel Joint Research Center for Precision Medicine Fudan University Shanghai 200438 China

Human Phenome Institute Fudan University Shanghai 201203 China

Illumina Inc 5200 Illumina Way San Diego CA 92122 USA

Immuneering Corporation One Broadway 14th Floor Cambridge MA 02142 USA

Institute for Molecular Medicine Finland Helsinki Finland

Institute of Medical and Molecular Genetics Hospital Universitario La Paz CIBERER Instituto de Salud Carlos 3 28046 Madrid Spain

JMP Life Sciences SAS Institute Inc Cary NC 27519 USA

Kelly Government Solutions Inc Research Triangle Park NC 27709 USA

Małopolska Centre of Biotechnology Jagiellonian University Krakow Poland

Market and Application Development Bioinformatics Roche Sequencing Solutions Inc 4300 Hacienda Dr Pleasanton CA 94588 USA

Marketing Integrated DNA Technologies Inc 1710 Commercial Park Coralville IA 52241 USA

Massachusetts General Hospital Harvard Medical School Boston MA 02114 USA

National Institute of Environmental Health Sciences Research Triangle Park Durham NC 27709 USA

Q2 Solutions EA Genomics 5927 S Miami Blvd Morrisville NC 27560 USA

Research and Development Burning Rock Biotech Shanghai 201114 China

Research and Development QIAGEN Sciences Inc Frederick MD 21703 USA

Research and Development Roche Sequencing Solutions Inc 500 South Rosa Rd Madison WI 53719 USA

Stanford Genome Technology Center Stanford University Palo Alto CA 94304 USA

State Key Laboratory of Genetic Engineering School of Life Sciences and Shanghai Cancer Hospital Cancer Institute Fudan University Shanghai 200438 China

Thermo Fisher Scientific 110 Miller Ave Ann Arbor MI 48104 USA

University of Arkansas at Little Rock Little Rock AR 72204 USA

University of North Carolina Health 101 Manning Drive Chapel Hill NC 27514 USA

University of Texas Southwestern Medical Center 2330 Inwood Rd Dallas TX 75390 USA

Winthrop P Rockefeller Cancer Institute University of Arkansas for Medical Sciences 4301 W Markham St Little Rock AR 72205 USA

Zobrazit více v PubMed

MAQC consortium. MicroArray/Sequencing Quality Control (MAQC/SEQC). U.S. Food and Drug Administration. 2019. https://www.fda.gov/science-research/bioinformatics-tools/microarraysequencing-quality-control-maqcseqc#MAQC_IV. Accessed 24 Feb 2020.

Zook JM, Catoe D, McDaniel J, Vang L, Spies N, Sidow A, Weng Z, Liu Y, Mason CE, Alexander N, Henaff E, McIntyre ABR, Chandramohan D, Chen F, Jaeger E, Moshrefi A, Pham K, Stedman W, Liang T, Saghbini M, Dzakula Z, Hastie A, Cao H, Deikus G, Schadt E, Sebra R, Bashir A, Truty RM, Chang CC, Gulbahce N, Zhao K, Ghosh S, Hyland F, Fu Y, Chaisson M, Xiao C, Trow J, Sherry ST, Zaranek AW, Ball M, Bobe J, Estep P, Church GM, Marks P, Kyriazopoulou-Panagiotopoulou S, Zheng GXY, Schnall-Levin M, Ordonez HS, Mudivarti PA, Giorda K, Sheng Y, Rypdal KB, Salit M. Extensive sequencing of seven human genomes to characterize benchmark reference materials. Sci Data. 2016;3(1):160025. doi: 10.1038/sdata.2016.25. PubMed DOI PMC

Suzuki T, Tsukumo Y, Furihata C, Naito M, Kohara A. Preparation of the standard cell lines for reference mutations in cancer gene-panels by genome editing in HEK 293 T/17 cells. Genes and Environ. 2020;42:8. 10.1186/s41021-020-0147-2. PubMed PMC

Craig DW, Nasser S, Corbett R, Chan SK, Murray L, Legendre C, Tembe W, Adkins J, Kim N, Wong S, Baker A, Enriquez D, Pond S, Pleasance E, Mungall AJ, Moore RA, McDaniel T, Ma Y, Jones SJM, Marra MA, Carpten JD, Liang WS. A somatic reference standard for cancer genome sequencing. Sci Rep. 2016;6(1):24607. doi: 10.1038/srep24607. PubMed DOI PMC

Kim J, Kim D, Lim JS, Maeng JH, Son H, Kang H-C, Nam H, Lee JH, Kim S. The use of technical replication for detection of low-level somatic mutations in next-generation sequencing. Nat Commun. 2019;10(1):1047. doi: 10.1038/s41467-019-09026-y. PubMed DOI PMC

Fang LT, SEQC2 Somatic Mutation Working Group. Establishing reference samples for detection of somatic mutations and germline variants with NGS technologies. bioRxiv. 2019. 10.1101/625624. Accessed 24 Feb 2020.

Horizon Discovery Ltd. Oncospan Reference Standard HD827. https://www.horizondiscovery.com/reference-standards/type/oncospan. Accessed 17 Apr. 2019.

Thermo Scientific. AcroMetrix Oncology Hotspot Control Package Insert. https://www.thermofisher.com/document-connect/document-connect.html?url=https%3A%2F%2Fassets.thermofisher.com%2FTFS-Assets%2FCDD%2Fmanuals%2FMAN0010820-AMX-Oncology-Hotspot-Ctrl-EN.pdf&title=QWNyb01ldHJpeCBPbmNvbG9neSBIb3RzcG90IENvbnRyb2wgUGFja2FnZSBJbnNlcnQgW0VOXQ==. Accessed 24 Apr. 2019.

MAQC Consortium The MicroArray quality control (MAQC) project shows inter- and intraplatform reproducibility of gene expression measurements. Nat Biotechnol. 2006;24(9):1151–1161. doi: 10.1038/nbt1239. PubMed DOI PMC

SEQC/MAQC-III Consortium, Su Z, Łabaj PP, Li S, Thierry-Mieg J, Thierry-Mieg D, et al. A comprehensive assessment of RNA-seq accuracy, reproducibility and information content by the Sequencing Quality Control Consortium. Nat Biotechnol. 2014;32(9):903–14. 10.1038/nbt.2957. PubMed PMC

MAQC Consortium, Shi L, Campbell G, Jones WD, Campagne F, Wen Z, et al. The MicroArray quality control (MAQC)-II study of common practices for the development and validation of microarray-based predictive models. Nat Biotechnol. 2010;28(8):827–38. 10.1038/nbt.1665. PubMed PMC

Shi L, Perkins RG, Fang H, Tong W. Reproducible and reliable microarray results through quality control: good laboratory proficiency and appropriate data analysis practices are essential. Curr Opin Biotechnol. 2008;19(1):10–18. doi: 10.1016/j.copbio.2007.11.003. PubMed DOI

Hong H, Shi L, Su Z, Ge W, Jones WD, Czika W, Miclaus K, Lambert CG, Vega SC, Zhang J, Ning B, Liu J, Green B, Xu L, Fang H, Perkins R, Lin SM, Jafari N, Park K, Ahn T, Chierici M, Furlanello C, Zhang L, Wolfinger RD, Goodsaid F, Tong W. Assessing sources of inconsistencies in genotypes and their effects on genome-wide association studies with HapMap samples. Pharmacogenomics J. 2010;10(4):364–374. doi: 10.1038/tpj.2010.24. PubMed DOI PMC

Novoradovskaya N, Whitfield ML, Basehore LS, Novoradovsky A, Pesich R, Usary J, Karaca M, Wong WK, Aprelikova O, Fero M, Perou CM, Botstein D, Braman J. Universal reference RNA as a standard for microarray experiments. BMC Genomics. 2004;5(1):20. doi: 10.1186/1471-2164-5-20. PubMed DOI PMC

Roche NimbleGen. SeqCap EZ MedExome Target Enrichment Kit. https://sequencing.roche.com/content/dam/rochesequence/US/Resources/PDFs/TargetEnrichment/Data%20Sheet%20-%20MedExome.pdf. Accessed 24 Feb 2020.

IDT. xGen hybridization capture of DNA libraries for NGS target enrichment. http://sfvideo.blob.core.windows.net/sitefinity/docs/default-source/protocol/xgen-hybridization-capture-of-dna-libraries.pdf?sfvrsn=ab880a07_12. Accessed 24 Feb 2020.

Agilent Technologies. SureSelectXT target enrichment system for Illumina paired-end multiplexed sequencing library protocol version C2, December 2018. https://www.agilent.com/cs/library/usermanuals/Public/G7530-90000.pdf. Accessed 24 Feb 2020.

Thermo Fisher Scientific. Ion AmpliSeq Exome RDY Kit. https://tools.thermofisher.com/content/sfs/brochures/Ion-AmpliSeq-Exome-Kit-Product-Flyer.pdf. Accessed 24 Feb. 2020.

10X Genomics. Chromium Genome Solution. http://go.10xgenomics.com/l/172142/2016-08-10/3svk9/172142/8086/LIT00003_RevB_Chromium_Genome_Solution_Application_Note_Digital.pdf. Accessed 24 Feb 2020.

McKenna A, Hanna M, Banks E, Sivachenko A, Cibulskis K, Kernytsky A, Garimella K, Altshuler D, Gabriel S, Daly M, DePristo MA. The genome analysis toolkit: a MapReduce framework for analyzing next-generation DNA sequencing data. Genome Res. 2010;20(9):1297–1303. doi: 10.1101/gr.107524.110. PubMed DOI PMC

Garrison E, Marth G. Haplotype-based variant detection from short-read sequencing. arXiv:1207.3907 [q-bio.GN]. Accessed 20 Mar 2019.

Cibulskis K, Lawrence MS, Carter SL, Sivachenko A, Jaffe D, Sougnez C, Gabriel S, Meyerson M, Lander ES, Getz G. Sensitive detection of somatic point mutations in impure and heterogeneous cancer samples. Nat Biotechnol. 2013;31(3):213–219. doi: 10.1038/nbt.2514. PubMed DOI PMC

Rimmer A, Phan H, Mathieson I, Iqbal Z, Twigg SRF. Wgs500 Consortium, et al. Integrating mapping-, assembly- and haplotype-based approaches for calling variants in clinical sequencing applications. Nat Genet. 2014;46(8):912–918. doi: 10.1038/ng.3036. PubMed DOI PMC

Li H, Handsaker B, Wysoker A, Fennell T, Ruan J, Homer N, Marth G, Abecasis G, Durbin R, 1000 Genome Project Data Processing Subgroup The Sequence Alignment/Map format and SAMtools. Bioinformatics. 2009;25(16):2078–2079. doi: 10.1093/bioinformatics/btp352. PubMed DOI PMC

Freed D, Pan R, Aldana R. TNscope: accurate detection of somatic mutations with haplotype-based variant candidate detection and machine learning filtering. bioRxiv. 2018. 10.1101/250647. Accessed 22 June 2018.

Lai Z, Markovets A, Ahdesmaki M, Chapman B, Hofmann O, McEwen R, Johnson J, Dougherty B, Barrett JC, Dry JR. VarDict: a novel and versatile variant caller for next-generation sequencing in cancer research. Nucleic Acids Res. 2016;44(11):e108. doi: 10.1093/nar/gkw227. PubMed DOI PMC

Koboldt DC, Chen K, Wylie T, Larson DE, McLellan MD, Mardis ER, et al. VarScan: variant detection in massively parallel sequencing of individual and pooled samples. Bioinformatics. 2009;25(17):2283–2285. doi: 10.1093/bioinformatics/btp373. PubMed DOI PMC

Fang LT, Afshar PT, Chhibber A, Mohiyuddin M, Fan Y, Mu JC, Gibeling G, Barr S, Asadi NB, Gerstein MB, Koboldt DC, Wang W, Wong WH, Lam HYK. An ensemble approach to accurately detect somatic mutations using SomaticSeq. Genome Biol. 2015;16(1):197. doi: 10.1186/s13059-015-0758-2. PubMed DOI PMC

Li H. Aligning sequence reads, clone sequences and assembly contigs with BWA-MEM. arXiv:1303.3997 [q-bio.GN]. Accessed 5 July 2018.

Li H, Durbin R. Fast and accurate short read alignment with Burrows–Wheeler transform. Bioinformatics. 2009;25(14):1754–1760. doi: 10.1093/bioinformatics/btp324. PubMed DOI PMC

Langmead B, Salzberg SL. Fast gapped-read alignment with Bowtie 2. Nat Methods. 2012;9(4):357–359. doi: 10.1038/nmeth.1923. PubMed DOI PMC

Morgulis A, Gertz EM, Schäffer AA, Agarwala R. A fast and symmetric DUST implementation to mask low-complexity DNA sequences. J Comput Biol. 2006;13(5):1028–1040. doi: 10.1089/cmb.2006.13.1028. PubMed DOI

Bamford S, Dawson E, Forbes S, Clements J, Pettett R, Dogan A, Flanagan A, Teague J, Futreal PA, Stratton MR, Wooster R. The COSMIC (Catalogue of Somatic Mutations in Cancer) database and website. Br J Cancer. 2004;91(2):355–358. doi: 10.1038/sj.bjc.6601894. PubMed DOI PMC

Pleasance ED, Cheetham RK, Stephens PJ, McBride DJ, Humphray SJ, Greenman CD, et al. A comprehensive catalogue of somatic mutations from a human cancer genome. Nature. 2010;463(7278):191–196. doi: 10.1038/nature08658. PubMed DOI PMC

Michor F, Polyak K. The origins and implications of Intratumor heterogeneity. Cancer Prev Res (Phila Pa) 2010;3(11):1361–1364. doi: 10.1158/1940-6207.CAPR-10-0234. PubMed DOI PMC

Wang VG, Kim H, Chuang JH. Whole-exome sequencing capture kit biases yield false negative mutation calls in TCGA cohorts. PLoS One. 2018;13(10):e0204912. doi: 10.1371/journal.pone.0204912. PubMed DOI PMC

Pagani F, Baralle FE. Genomic variants in exons and introns: identifying the splicing spoilers. Nat Rev Genet. 2004;5(5):389–396. doi: 10.1038/nrg1327. PubMed DOI

Spatz A, Borg C, Feunteun J. X-chromosome genetics and human cancer. Nat Rev Cancer. 2004;4(8):617–629. doi: 10.1038/nrc1413. PubMed DOI

Xiao W, SEQC2 Somatic Mutation Working Group. Achieving reproducibility and accuracy in cancer mutation detection with whole-genome and whole-exome sequencing. bioRxiv. 2019. 10.1101/626440. Accessed 24 Feb 2020.

Shigemizu D, Momozawa Y, Abe T, Morizono T, Boroevich KA, Takata S, Ashikawa K, Kubo M, Tsunoda T. Performance comparison of four commercial human whole-exome capture platforms. Sci Rep. 2015;5(1):12742. doi: 10.1038/srep12742. PubMed DOI PMC

Belkadi A, Bolze A, Itan Y, Cobat A, Vincent QB, Antipenko A, Shang L, Boisson B, Casanova JL, Abel L. Whole-genome sequencing is more powerful than whole-exome sequencing for detecting exome variants. Proc Natl Acad Sci. 2015;112(17):5473–5478. doi: 10.1073/pnas.1418631112. PubMed DOI PMC

Tate JG, Bamford S, Jubb HC, Sondka Z, Beare DM, Bindal N, Boutselakis H, Cole CG, Creatore C, Dawson E, Fish P, Harsha B, Hathaway C, Jupe SC, Kok CY, Noble K, Ponting L, Ramshaw CC, Rye CE, Speedy HE, Stefancsik R, Thompson SL, Wang S, Ward S, Campbell PJ, Forbes SA. COSMIC: the catalogue of somatic mutations in cancer. Nucleic Acids Res. 2019;47(D1):D941–D947. doi: 10.1093/nar/gky1015. PubMed DOI PMC

Zook JM, Chapman B, Wang J, Mittelman D, Hofmann O, Hide W, Salit M. Integrating human sequence data sets provides a resource of benchmark SNP and indel genotype calls. Nat Biotechnol. 2014;32(3):246–251. doi: 10.1038/nbt.2835. PubMed DOI

Zook JM, McDaniel J, Olson ND, Wagner J, Parikh H, Heaton H, et al. An open resource for accurately benchmarking small variant and reference calls. Nat Biotechnol. 2019;37(5):561–6. 10.1038/s41587-019-0074-6. PubMed PMC

Wagner J, Olson ND, Harris L, Khan Z, Farek J, Mahmoud M, et al. Benchmarking challenging small variants with linked and long reads. bioRxiv. 2020. 10.1101/2020.07.24.212712. Accessed 24 Feb 2020. PubMed PMC

Gong B, SEQC2 Oncopanel Sequencing Working Group. Cross-oncopanel study reveals high sensitivity and accuracy with overall analytical performance depending on genomic regions. Genome Biol. 10.1186/s13059-021-02315-0. PubMed PMC

Devason I, SEQC2 Oncopanel Sequencing Working Group. Evaluating the analytical validity of circulating tumor DNA sequencing assays for precision oncology. Nat Biotechnol. 10.1038/s41587-021-00857-z. PubMed PMC

Fisher S, Barry A, Abreu J, Minie B, Nolan J, Delorey TM, Young G, Fennell TJ, Allen A, Ambrogio L, Berlin AM, Blumenstiel B, Cibulskis K, Friedrich D, Johnson R, Juhn F, Reilly B, Shammas R, Stalker J, Sykes SM, Thompson J, Walsh J, Zimmer A, Zwirko Z, Gabriel S, Nicol R, Nusbaum C. A scalable, fully automated process for construction of sequence-ready human exome targeted capture libraries. Genome Biol. 2011;12(1):R1. doi: 10.1186/gb-2011-12-1-r1. PubMed DOI PMC

Thermo Fisher Scientific. Ion AmpliSeq Exome RDY Kit 1x8. https://www.thermofisher.com/order/catalog/product/A38262?SID=srch-srp-A38262. Accessed 16 Oct. 2019.

Thermo Fisher Scientific. Ion AmpliSeq™ Exome RDY Library Preparation User Guide - MAN0010084. https://assets.thermofisher.com/TFS-Assets/LSG/manuals/MAN0010084_AmpliSeq_ExomeRDY_LibraryPrep_UG.pdf. Accessed 16 Oct. 2019.

Thermo Fisher Scientific, "IonCode™ Barcode Adapters 1–384 Kit - A29751. https://www.thermofisher.com/order/catalog/product/A29751. Accessed 16 Oct. 2019.

Thermo Fisher Scientific, "Ion 540™ Kit-Chef - A30011. https://www.thermofisher.com/order/catalog/product/A30011?SID=srch-srp-A30011. Accessed 16 Oct. 2019.

Thermo Fisher Scientific, "Ion S5™ XL System - A27214. https://www.thermofisher.com/order/catalog/product/A27214?SID=srch-srp-A27214. Accessed 16 Oct. 2019.

Thermo Fisher Scientific, "Ion 540™ Chip Kit - A27766. https://www.thermofisher.com/order/catalog/product/A27765?SID=srch-srp-A27765. Accessed 16 Oct. 2019.

Rothberg JM, Hinz W, Rearick TM, Schultz J, Mileski W, Davey M, Leamon JH, Johnson K, Milgrew MJ, Edwards M, Hoon J, Simons JF, Marran D, Myers JW, Davidson JF, Branting A, Nobile JR, Puc BP, Light D, Clark TA, Huber M, Branciforte JT, Stoner IB, Cawley SE, Lyons M, Fu Y, Homer N, Sedova M, Miao X, Reed B, Sabina J, Feierstein E, Schorn M, Alanjary M, Dimalanta E, Dressman D, Kasinskas R, Sokolsky T, Fidanza JA, Namsaraev E, McKernan KJ, Williams A, Roth GT, Bustillo J. An integrated semiconductor device enabling non-optical genome sequencing. Nature. 2011;475(7356):348–352. doi: 10.1038/nature10242. PubMed DOI

Picard Tools - By Broad Institute. http://broadinstitute.github.io/picard/. Accessed 22 Dec. 2017.

Narasimhan V, Danecek P, Scally A, Xue Y, Tyler-Smith C, Durbin R. BCFtools/RoH: a hidden Markov model approach for detecting autozygosity from next-generation sequencing data. Bioinformatics. 2016;32(11):1749–1751. doi: 10.1093/bioinformatics/btw044. PubMed DOI PMC

DePristo MA, Banks E, Poplin R, Garimella KV, Maguire JR, Hartl C, et al. A framework for variation discovery and genotyping using next-generation DNA sequencing data. Nat Genet. 2011;43(5):491–498. doi: 10.1038/ng.806. PubMed DOI PMC

Babraham Bioinformatics group. FastQC A Quality Control tool for High Throughput Sequence Data. https://www.bioinformatics.babraham.ac.uk/projects/fastqc/. Accessed 4 Sept 2018.

Criscuolo A, Brisse S. AlienTrimmer: A tool to quickly and accurately trim off multiple short contaminant sequences from high-throughput sequencing reads. Genomics. 2013;102(5–6):500–506. doi: 10.1016/j.ygeno.2013.07.011. PubMed DOI

Koboldt DC, Zhang Q, Larson DE, Shen D, McLellan MD, Lin L, et al. VarScan 2: somatic mutation and copy number alteration discovery in cancer by exome sequencing. Genome Res. 2012;22(3):568–576. doi: 10.1101/gr.129684.111. PubMed DOI PMC

Martin M. Cutadapt removes adapter sequences from high-throughput sequencing reads. EMBnet. J. 2011;17(1):10–12. doi: 10.14806/ej.17.1.200. DOI

Bolger AM, Lohse M, Usadel B. Trimmomatic: a flexible trimmer for Illumina sequence data. Bioinformatics. 2014;30(15):2114–2120. doi: 10.1093/bioinformatics/btu170. PubMed DOI PMC

Wang K, Li M, Hakonarson H. ANNOVAR: functional annotation of genetic variants from high-throughput sequencing data. Nucleic Acids Res. 2010;38(16):e164. doi: 10.1093/nar/gkq603. PubMed DOI PMC

Liu X, Wu C, Li C, Boerwinkle E. dbNSFP v3.0: a one-stop database of functional predictions and annotations for human nonsynonymous and splice-site SNVs. Hum Mutat. 2016;37(3):235–241. doi: 10.1002/humu.22932. PubMed DOI PMC

Lek M, Karczewski KJ, Minikel EV, Samocha KE, Banks E, Fennell T, et al. Analysis of protein-coding genetic variation in 60,706 humans. Nature. 2016;536(7616):285–291. doi: 10.1038/nature19057. PubMed DOI PMC

Mose LE, Wilkerson MD, Hayes DN, Perou CM, Parker JS. ABRA: improved coding indel detection via assembly-based realignment. Bioinformatics. 2014;30(19):2813–2815. doi: 10.1093/bioinformatics/btu376. PubMed DOI PMC

Freed D, Aldana R, Weber JA, Edwards JS. The Sentieon Genomics Tools - A fast and accurate solution to variant calling from next-generation sequence data. bioRxiv. 2017. 10.1101/115717. Accessed 22 June 2018.

Soong D, Stratford J, Avet-Loiseau H, Bahlis N, Davies F, Dispenzieri A, Sasser AK, Schecter JM, Qi M, Brown C, Jones W, Keats JJ, Auclair D, Chiu C, Powers J, Schaffer M. CNV radar: an improved method for somatic copy number alteration characterization in oncology. BMC Bioinformatics. 2020;21(1):98. doi: 10.1186/s12859-020-3397-x. PubMed DOI PMC

Sturm M, Schroeder C, Bauer P. SeqPurge: highly-sensitive adapter trimming for paired-end NGS data. BMC Bioinformatics. 2016;17(1):1–7. doi: 10.1186/s12859-016-1069-7. PubMed DOI PMC

Kim D, Langmead B, Salzberg SL. HISAT: a fast spliced aligner with low memory requirements. Nat Methods. 2015;12(4):357–360. doi: 10.1038/nmeth.3317. PubMed DOI PMC

Real Time Genomics (RTG) Variant Caller. https://www.realtimegenomics.com/. Accessed 24 Feb 2020.

Chen K, Wallis JW, McLellan MD, Larson DE, Kalicki JM, Pohl CS, et al. BreakDancer: an algorithm for high-resolution mapping of genomic structural variation. Nat Methods. 2009;6(9):677–681. doi: 10.1038/nmeth.1363. PubMed DOI PMC

Abyzov A, Urban AE, Snyder M, Gerstein M. CNVnator: an approach to discover, genotype, and characterize typical and atypical CNVs from family and population genome sequencing. Genome Res. 2011;21(6):974–984. doi: 10.1101/gr.114876.110. PubMed DOI PMC

Rausch T, Zichner T, Schlattl A, Stütz AM, Benes V, Korbel JO. DELLY: structural variant discovery by integrated paired-end and split-read analysis. Bioinformatics. 2012;28(18):i333–i339. doi: 10.1093/bioinformatics/bts378. PubMed DOI PMC

Handsaker RE, Van Doren V, Berman JR, Genovese G, Kashin S, Boettger LM, et al. Large multiallelic copy number variations in humans. Nat Genet. 2015;47(3):296–303. doi: 10.1038/ng.3200. PubMed DOI PMC

Yang L, Luquette LJ, Gehlenborg N, Xi R, Haseley PS, Hsieh C-H, Zhang C, Ren X, Protopopov A, Chin L, Kucherlapati R, Lee C, Park PJ. Diverse mechanisms of somatic structural variations in human Cancer genomes. Cell. 2013;153(4):919–929. doi: 10.1016/j.cell.2013.04.010. PubMed DOI PMC

Mohiyuddin M, Mu JC, Li J, Bani Asadi N, Gerstein MB, Abyzov A, Wong WH, Lam HYK. MetaSV: an accurate and integrative structural-variant caller for next generation sequencing. Bioinformatics. 2015;31(16):2741–2744. doi: 10.1093/bioinformatics/btv204. PubMed DOI PMC

Parikh H, Mohiyuddin M, Lam HYK, Iyer H, Chen D, Pratt M, et al. svclassify: a method to establish benchmark structural variant calls. BMC Genomics. 2016;17(1):64. doi: 10.1186/s12864-016-2366-2. PubMed DOI PMC

Ye K, Schulz MH, Long Q, Apweiler R, Ning Z. Pindel: a pattern growth approach to detect break points of large deletions and medium sized insertions from paired-end short reads. Bioinformatics. 2009;25(21):2865–2871. doi: 10.1093/bioinformatics/btp394. PubMed DOI PMC

Talevich E, Shain AH, Botton T, Bastian BC. CNVkit: genome-wide copy number detection and visualization from targeted DNA sequencing. PLoS Comput Biol. 2016;12(4):e1004873. doi: 10.1371/journal.pcbi.1004873. PubMed DOI PMC

Jiang Y, Oldridge DA, Diskin SJ, Zhang NR. CODEX: a normalization and copy number variation detection method for whole exome sequencing. Nucleic Acids Res. 2015;43(6):e39. doi: 10.1093/nar/gku1363. PubMed DOI PMC

Kuilman T, Velds A, Kemper K, Ranzani M, Bombardelli L, Hoogstraat M, Nevedomskaya E, Xu G, de Ruiter J, Lolkema MP, Ylstra B, Jonkers J, Rottenberg S, Wessels LF, Adams DJ, Peeper DS, Krijgsman O. CopywriteR: DNA copy number detection from off-target sequence data. Genome Biol. 2015;16(1):49. doi: 10.1186/s13059-015-0617-1. PubMed DOI PMC

Zhang Y, Yu Z, Ban R, Zhang H, Iqbal F, Zhao A, Li A, Shi Q. DeAnnCNV: a tool for online detection and annotation of copy number variations from whole-exome sequencing data. Nucleic Acids Res. 2015;43(W1):W289–W294. doi: 10.1093/nar/gkv556. PubMed DOI PMC

Magi A, Tattini L, Cifola I, D’Aurizio R, Benelli M, Mangano E, Battaglia C, Bonora E, Kurg A, Seri M, Magini P, Giusti B, Romeo G, Pippucci T, Bellis GD, Abbate R, Gensini GF. EXCAVATOR: detecting copy number variants from whole-exome sequencing data. Genome Biol. 2013;14(10):R120. doi: 10.1186/gb-2013-14-10-r120. PubMed DOI PMC

Plagnol V, Curtis J, Epstein M, Mok KY, Stebbings E, Grigoriadou S, Wood NW, Hambleton S, Burns SO, Thrasher AJ, Kumararatne D, Doffinger R, Nejentsev S. A robust model for read count data in exome sequencing experiments and implications for copy number variant calling. Bioinformatics. 2012;28(21):2747–2754. doi: 10.1093/bioinformatics/bts526. PubMed DOI PMC

Chang L-C, Das B, Lih C-J, Si H, Camalier CE, McGregor PM, et al. RefCNV: identification of gene-based copy number variants using whole exome sequencing. Cancer Inform. 2016;15:65–71. 10.4137/CIN.S36612. PubMed PMC

Zhang Z, Hao K. SAAS-CNV: a joint segmentation approach on aggregated and allele specific signals for the identification of somatic copy number alterations with next-generation sequencing data. PLoS Comput Biol. 2015;11(11):e1004618. doi: 10.1371/journal.pcbi.1004618. PubMed DOI PMC

Thermo Fisher Scientific. Torrent Suite Software. https://github.com/iontorrent/TS. Accessed 16 Oct 2019.

Thermo Fisher Scientific. Ion Reporter Software. https://www.thermofisher.com/us/en/home/life-science/sequencing/next-generation-sequencing/ion-torrent-next-generation-sequencing-workflow/ion-torrent-next-generation-sequencing-data-analysis-workflow/ion-reporter-software.html. Accessed 16 Oct. 2019.

Thermo Fisher Scientific. TMAP - Torrent Mapper. https://github.com/iontorrent/TS. Accessed 16 Oct. 2019.

Thermo Fisher Scientific. Torrent Variant Caller. http://updates.iontorrent.com/tvc_standalone/. Accessed 16 Oct 2019.

Kim S, Scheffler K, Halpern AL, Bekritsky MA, Noh E, Källberg M, et al. Strelka2: fast and accurate calling of germline and somatic variants. Nat Methods. 2018;15(8):591–4. 10.1038/s41592-018-0051-x. PubMed

Broad Institute. MuTect2. https://software.broadinstitute.org/gatk/documentation/tooldocs/3.8-0/org_broadinstitute_gatk_tools_walkers_cancer_m2_MuTect2.php. Accessed 24 Feb 2020.

Wilm A, Aw PPK, Bertrand D, Yeo GHT, Ong SH, Wong CH, Khor CC, Petric R, Hibberd ML, Nagarajan N. LoFreq: a sequence-quality aware, ultra-sensitive variant caller for uncovering cell-population heterogeneity from high-throughput sequencing datasets. Nucleic Acids Res. 2012;40(22):11189–11201. doi: 10.1093/nar/gks918. PubMed DOI PMC

Narzisi G, O’Rawe JA, Iossifov I, Fang H, Lee Y, Wang Z, et al. Accurate de novo and transmitted indel detection in exome-capture data using microassembly. Nat Methods. 2014;11(10):1033–1036. doi: 10.1038/nmeth.3069. PubMed DOI PMC

SEQC2 Onco-panel Sequencing Working Group. A verified genomic reference sample for assessing performance of variant calling. figshare. 2021. 10.6084/m9.figshare.13511829. Accessed 25 Feb 2021.

Quinlan AR, Hall IM. BEDTools: a flexible suite of utilities for comparing genomic features. Bioinformatics. 2010;26(6):841–842. doi: 10.1093/bioinformatics/btq033. PubMed DOI PMC

Krusche P, Trigg L, Boutros PC, Mason CE, Vega FMDL, Moore BL, et al. Best practices for benchmarking germline small-variant calls in human genomes. Nat Biotechnol. 2019;37(5):555–560. doi: 10.1038/s41587-019-0054-x. PubMed DOI PMC

Lawrence M, Gentleman R, Carey V. rtracklayer: an R package for interfacing with genome browsers. Bioinformatics. 2009;25(14):1841–1842. doi: 10.1093/bioinformatics/btp328. PubMed DOI PMC

Hindson BJ, Ness KD, Masquelier DA, Belgrader P, Heredia NJ, Makarewicz AJ, Bright IJ, Lucero MY, Hiddessen AL, Legler TC, Kitano TK, Hodel MR, Petersen JF, Wyatt PW, Steenblock ER, Shah PH, Bousse LJ, Troup CB, Mellen JC, Wittmann DK, Erndt NG, Cauley TH, Koehler RT, So AP, Dube S, Rose KA, Montesclaros L, Wang S, Stumbo DP, Hodges SP, Romine S, Milanovich FP, White HE, Regan JF, Karlin-Neumann GA, Hindson CM, Saxonov S, Colston BW. High-throughput droplet digital PCR system for absolute quantitation of DNA copy number. Anal Chem. 2011;83(22):8604–8610. doi: 10.1021/ac202028g. PubMed DOI PMC

Willey JC, Morrison T, Austermiller B, Crawford EL, Craig DJ, Blomquist T, et al. Assessing synthetic reference sequence internal standards as quality-control for NGS measurement of actionable mutations in circulating tumor DNA. Cell Genomics. Submitted.

SEQC2 Onco-panel Sequencing Working Group. Genomic Reference Material for Assessing Performance of mutation detection. BioProject PRJNA673156. NCBI. 2021. https://www.ncbi.nlm.nih.gov/bioproject/PRJNA673156. Accessed 26 Feb 2021.

Deep oncopanel sequencing reveals within block position-dependent quality degradation in FFPE processed samples

Evaluating the analytical validity of circulating tumor DNA sequencing assays for precision oncology

Cross-oncopanel study reveals high sensitivity and accuracy with overall analytical performance depending on genomic regions