One of the most common statistical analyses in experimental psychology concerns the comparison of two means using the frequentist t test. However, frequentist t tests do not quantify evidence and require various assumption tests. Recently, popularized Bayesian t tests do quantify evidence, but these were developed for scenarios where the two populations are assumed to have the same variance. As an alternative to both methods, we outline a comprehensive t test framework based on Bayesian model averaging. This new t test framework simultaneously takes into account models that assume equal and unequal variances, and models that use t-likelihoods to improve robustness to outliers. The resulting inference is based on a weighted average across the entire model ensemble, with higher weights assigned to models that predicted the observed data well. This new t test framework provides an integrated approach to assumption checks and inference by applying a series of pertinent models to the data simultaneously rather than sequentially. The integrated Bayesian model-averaged t tests achieve robustness without having to commit to a single model following a series of assumption checks. To facilitate practical applications, we provide user-friendly implementations in JASP and via the RoBTT package in R . A tutorial video is available at https://www.youtube.com/watch?v=EcuzGTIcorQ.

Department of Experimental Psychology University College London 26 Bedford Way 129 B WC1H 0AP London UK

Department of Psychology University of Amsterdam Amsterdam The Netherlands

Department of Psychology University of Oslo Oslo Norway

Institute for Advanced Study University of Amsterdam Amsterdam Netherlands

Institute for Biodiversity and Ecosystem Dynamics University of Amsterdam Amsterdam Netherlands

Institute of Computer Science Czech Academy of Sciences Prague Czechia

KG Jebsen Centre for Neurodevelopmental Disorders University of Oslo Oslo Norway

Machine Learning Group CWI Amsterdam Amsterdam The Netherlands

NevSom Department of Rare Disorders Oslo University Hospital Oslo Norway

References provided by Crossref.org