Footprint of publication selection bias on meta-analyses in medicine, environmental sciences, psychology, and economics - Bartoš, František; Maier, Maximilian; Wagenmakers, Eric-Jan; Nippold, Franziska; Doucouliagos, Hristos; Ioannidis, John P A; Otte, Willem M; Sladekova, Martina; Deresssa, Teshome K; Bruns, Stephan B; Fanelli, Daniele; Stanley, T D (Research synthesis methods, 2024)
Publication selection bias undermines the systematic accumulation of evidence. To assess the extent of this problem, we survey over 68,000 meta-analyses containing over 700,000 effect size estimates from medicine (67,386/597,699), environmental sciences (199/12,707), psychology (605/23,563), and economics (327/91,421). Our results indicate that meta-analyses in economics are the most severely contaminated by publication selection bias, closely followed by meta-analyses in environmental sciences and psychology, whereas meta-analyses in medicine are contaminated the least. After adjusting for publication selection bias, the median probability of the presence of an effect decreased from 99.9% to 29.7% in economics, from 98.9% to 55.7% in psychology, from 99.8% to 70.7% in environmental sciences, and from 38.0% to 29.7% in medicine. The median absolute effect sizes (in terms of standardized mean differences) decreased from d = 0.20 to d = 0.07 in economics, from d = 0.37 to d = 0.26 in psychology, from d = 0.62 to d = 0.43 in environmental sciences, and from d = 0.24 to d = 0.13 in medicine.
Keywords: lékařství; výběrový bias; Meta-Analysis as Topic; psychologie; ekonomika; Medicine; publikační zkreslení; Selection Bias; model‐averaging; Psychology; Economics; metaanalýza jako téma; Ecology; Publication Bias; publication bias; evidence; výzkumný projekt; lidé; Research Design; ekologie; Humans; meta‐analysis; RoBMA; effect sizes; pravděpodobnost; Bayesian; Probability