One of the aims of high-throughput gene/protein profiling experiments is the identification of biological processes altered between two or more conditions. Pathway analysis is an umbrella term for a multitude of computational approaches used for this purpose. While in the beginning pathway analysis relied on enrichment-based approaches, a newer generation of methods is now available, exploiting pathway topologies in addition to gene/protein expression levels. However, little effort has been invested in their critical assessment with respect to their performance in different experimental setups. Here, we assessed the performance of seven representative methods identifying differentially expressed pathways between two groups of interest based on gene expression data with prior knowledge of pathway topologies: SPIA, PRS, CePa, TAPPA, TopologyGSA, Clipper and DEGraph. We performed a number of controlled experiments that investigated their sensitivity to sample and pathway size, threshold-based filtering of differentially expressed genes, ability to detect target pathways, ability to exploit the topological information and the sensitivity to different pre-processing strategies. We also verified type I error rates and described the influence of overexpression of single genes, gene sets and topological motifs of various sizes on the detection of a pathway as differentially expressed. The results of our experiments demonstrate a wide variability of the tested methods. We provide a set of recommendations for an informed selection of the proper method for a given data analysis task.

Institute of Biostatistics and Analyses Faculty of Medicine Masarykova Univerzita Brno Czech Republic

RECETOX Faculty of Science Masarykova Univerzita Brno Czech Republic

Zobrazit více v PubMed

Subramanian A, Tamayo P, Mootha VK, Mukherjee S, Ebert BL, Gillette MA, et al. Gene set enrichment analysis: A knowledge-based approach for interpreting genome-wide expression profiles. Proceedings of the National Academy of Sciences of the United States of America. 2005;102(43):15545–15550. doi: 10.1073/pnas.0506580102 PubMed DOI PMC

Khatri P, Sirota M, Butte AJ. Ten Years of Pathway Analysis: Current Approaches and Outstanding Challenges. PLoS Comput Biol. 2012;8(2):e1002375 doi: 10.1371/journal.pcbi.1002375 PubMed DOI PMC

Emmert-Streib F, Tripathi S, Matos Simoes Rd. Harnessing the complexity of gene expression data from cancer: from single gene to structural pathway methods. Biology Direct. 2012;7(1):44 doi: 10.1186/1745-6150-7-44 PubMed DOI PMC

García-Campos MA, Espinal-Enríquez J, Hernández-Lemus E. Pathway Analysis: State of the Art. Frontiers in Physiology. 2015;6:383 doi: 10.3389/fphys.2015.00383 PubMed DOI PMC

Bayerlová M, Jung K, Kramer F, Klemm F, Bleckmann A, Beißbarth T. Comparative study on gene set and pathway topology-based enrichment methods. BMC Bioinformatics. 2015;16(1):334 doi: 10.1186/s12859-015-0751-5 PubMed DOI PMC

Braun R, Shah S. Network Methods for Pathway Analysis of Genomic Data; 2015.

Gu Z, Liu J, Cao K, Zhang J, Wang J. Centrality-based pathway enrichment: a systematic approach for finding significant pathways dominated by key genes. BMC Systems Biology. 2012;6(1):56 doi: 10.1186/1752-0509-6-56 PubMed DOI PMC

Mitrea C, Taghavi Z, Bokanizad B, Hanoudi S, Tagett R, Donato M, et al. Methods and approaches in the topology-based analysis of biological pathways. Frontiers in Physiology. 2013;4:278 doi: 10.3389/fphys.2013.00278 PubMed DOI PMC

Tarca AL, Draghici S, Khatri P, Hassan SS, Mittal P, Kim Js, et al. A novel signaling pathway impact analysis. Bioinformatics. 2009;25(1):75–82. doi: 10.1093/bioinformatics/btn577 PubMed DOI PMC

Al-Haj Ibrahim M, Jassim S, Cawthorne MA, Langlands K. A Topology-Based Score for Pathway Enrichment. J Comput Biol. 2012;. PubMed

Gao S, Wang X. TAPPA: topological analysis of pathway phenotype association. Bioinformatics. 2007;23(22):3100–3102. doi: 10.1093/bioinformatics/btm460 PubMed DOI PMC

Massa M, Chiogna M, Romualdi C. Gene set analysis exploiting the topology of a pathway. BMC Systems Biology. 2010;4(1):121 doi: 10.1186/1752-0509-4-121 PubMed DOI PMC

Martini P, Sales G, Massa MS, Chiogna M, Romualdi C. Along signal paths: an empirical gene set approach exploiting pathway topology. Nucleic Acids Research. 2012; doi: 10.1093/nar/gks866 PubMed DOI PMC

Jacob L, Neuvial P, Dudoit S. Gains in Power from Structured Two-Sample Tests of Means on Graphs. ArXiv e-prints. 2010;.

Smyth GK. Linear models and empirical bayes methods for assessing differential expression in microarray experiments. Statistical applications in genetics and molecular biology. 2004;3(1). doi: 10.2202/1544-6115.1027 PubMed DOI

R Core Team. R: A Language and Environment for Statistical Computing; 2014. Available from: http://www.R-project.org/.

Huber W, Carey J V, Gentleman R, et al. Orchestrating high-throughput genomic analysis with Bioconductor. Nature Methods. 2015;12(2):115–121. doi: 10.1038/nmeth.3252 PubMed DOI PMC

Sales G, Calura E, Romualdi C. graphite: GRAPH Interaction from pathway Topological Environment; 2016.

Ihnatova I, Budinska E. ToPASeq: an R package for topology-based pathway analysis of microarray and RNA-Seq data. BMC Bioinformatics. 2015;16(1):350 doi: 10.1186/s12859-015-0763-1 PubMed DOI PMC

Draghici S, Khatri P, Tarca AL, Amin K, Done A, Voichita C, et al. A systems biology approach for pathway level analysis. Genome Research. 2007;17(10):000 doi: 10.1101/gr.6202607 PubMed DOI PMC

Khatri P, Draghici S, Tarca AL, Hassan SS, Romero R. A system biology approach for the steady-state analysis of gene signaling networks. In: Proceedings of the Congress on pattern recognition 12th Iberoamerican conference on Progress in pattern recognition, image analysis and applications. CIARP’07. Berlin, Heidelberg: Springer-Verlag; 2007. p. 32–41. Available from: http://dl.acm.org/citation.cfm?id=1782914.1782919.

Junker BH, Schreiber F. Analysis of Biological Networks Wiley Series in Bioinformatics. Wiley; 2011. Available from: https://books.google.cz/books?id=YeXLbClh1SIC.

Kim JW, Mori S, Nevins JR. Myc-Induced MicroRNAs Integrate Myc-Mediated Cell Proliferation and Cell Fate. Cancer Research. 2010;70(12):4820–4828. doi: 10.1158/0008-5472.CAN-10-0659 PubMed DOI PMC

Bild AH, Yao G, Chang JT, Wang Q, Potti A, Chasse D, et al. Oncogenic pathway signatures in human cancers as a guide to targeted therapies. Nature. 2006;439(7074):353–7. doi: 10.1038/nature04296 PubMed DOI

Haibe-Kains B, Desmedt C, Loi S, Culhane AC, Bontempi G, Quackenbush J, et al. A three-gene model to robustly identify breast cancer molecular subtypes. Journal of the National Cancer Institute. 2012;104(4):311–325. doi: 10.1093/jnci/djr545 PubMed DOI PMC

Bhatti G, Tarca AL. KEGGdzPathwaysGEO: KEGG Disease Datasets from GEO; 2012.

Bhatti G. KEGGandMetacoreDzPathwaysGEO: Disease Datasets from GEO; 2014.

Sales G, Calura E, Cavalieri D, Romualdi C. graphite—a Bioconductor package to convert pathway topology to gene network. BMC Bioinformatics. 2012;13(1):20 doi: 10.1186/1471-2105-13-20 PubMed DOI PMC

Tripathi S, Emmert-Streib F. Assessment Method for a Power Analysis to Identify Differentially Expressed Pathways. PLOS ONE. 2012;7(5):1–13. doi: 10.1371/journal.pone.0037510 PubMed DOI PMC

Karlin S, Mrázek J, Campbell A, Kaiser D. Characterizations of Highly Expressed Genes of Four Fast-Growing Bacteria. J Bacteriol. 2001;183(17):5025–5040. doi: 10.1128/JB.183.17.5025-5040.2001 PubMed DOI PMC