Although variation in effect sizes and predicted values among studies of similar phenomena is inevitable, such variation far exceeds what might be produced by sampling error alone. One possible explanation for variation among results is differences among researchers in the decisions they make regarding statistical analyses. A growing array of studies has explored this analytical variability in different fields and has found substantial variability among results despite analysts having the same data and research question. Many of these studies have been in the social sciences, but one small "many analyst" study found similar variability in ecology. We expanded the scope of this prior work by implementing a large-scale empirical exploration of the variation in effect sizes and model predictions generated by the analytical decisions of different researchers in ecology and evolutionary biology. We used two unpublished datasets, one from evolutionary ecology (blue tit, Cyanistes caeruleus, to compare sibling number and nestling growth) and one from conservation ecology (Eucalyptus, to compare grass cover and tree seedling recruitment). The project leaders recruited 174 analyst teams, comprising 246 analysts, to investigate the answers to prespecified research questions. Analyses conducted by these teams yielded 141 usable effects (compatible with our meta-analyses and with all necessary information provided) for the blue tit dataset, and 85 usable effects for the Eucalyptus dataset. We found substantial heterogeneity among results for both datasets, although the patterns of variation differed between them. For the blue tit analyses, the average effect was convincingly negative, with less growth for nestlings living with more siblings, but there was near continuous variation in effect size from large negative effects to effects near zero, and even effects crossing the traditional threshold of statistical significance in the opposite direction. In contrast, the average relationship between grass cover and Eucalyptus seedling number was only slightly negative and not convincingly different from zero, and most effects ranged from weakly negative to weakly positive, with about a third of effects crossing the traditional threshold of significance in one direction or the other. However, there were also several striking outliers in the Eucalyptus dataset, with effects far from zero. For both datasets, we found substantial variation in the variable selection and random effects structures among analyses, as well as in the ratings of the analytical methods by peer reviewers, but we found no strong relationship between any of these and deviation from the meta-analytic mean. In other words, analyses with results that were far from the mean were no more or less likely to have dissimilar variable sets, use random effects in their models, or receive poor peer reviews than those analyses that found results that were close to the mean. The existence of substantial variability among analysis outcomes raises important questions about how ecologists and evolutionary biologists should interpret published results, and how they should conduct analyses in the future.

Animal Ecology and Tropical Biology University of Würzburg Biocenter Am Hubland 97074 Würzburg Germany

Apicultural State Institute University of Hohenheim Erna Hruschka Weg 6 70599 Stuttgart Germany

Applied Zoology Zellescher Weg 20b 01217 Dresden TUDresden Germany

Aquatic Ecology and Evolution Group Limnological Institute University of Konstanz Mainaustraße 252 78464 Konstanz Germany

Arthur Rylah Institute for Environmental Research 123 Brown Street Heidelberg Victoria 3084 Australia

Bangor University Bangor University Deiniol Road Bangor LL57 2UW UK

Baruch Marine Field Laboratory University of South Carolina 2306 Crabhaul Rd Georgetown SC 29440 USA

Biodiversity and Conservation Area Rey Juan Carlos University C Tulipán s n 28933 Móstoles Madrid Spain

Biodiversity Faculty of Forest Sciences and Forest Ecology University of Göttingen Macroecology and BiogeographyBüsgenweg 1 37077 Göttingen Germany

Biological and Environmental Sciences and Gothenburg Global Biodiversity Centre University of Gothenburg Medicinaregatan 7B SE 413 90 Gothenburg Sweden

Biological and Environmental Sciences University of Stirling Cottrell Building Stirling FK9 4LA UK

Biological Sciences Eastern Illinois University 600 Lincoln Avenue Charleston IL 61920 USA

Biology Department and Environmental Studies Department St Olaf College 1520 St Olaf Ave Northfield MN 55057 USA

Biology Department Brigham Young University 4102 Life Science Building Provo UT USA

Biology Department Lund University Sölvegatan 37 22362 Lund Sweden

Biology Department Technische Universität Darmstadt Schnittspahnstraße 3 64287 Darmstadt Germany

Biology Department University of Wisconsin La Crosse 1725 State St La Crosse WI 54601 USA

Biology Department Wake Forest University 1834 Wake Forest Rd Winston Salem NC 27109 USA

Biology Indiana University Purdue University Indianapolis 420 University Blvd Indianapolis IN 46202 USA

Biology University of Saskatchewan University of Saskatchewan 112 Science Place Saskatoon SK S7N 5E2 Canada

Biology University of Waterloo 200 University Ave W Waterloo Ontario N2L 3G1 Canada

BirdLife International David Attenborough Building Pembroke Street Cambridge CB2 3QZ UK

British Trust for Ornithology BTO The Nunnery Thetford Norfolk IP24 2PU UK

CABI Bakeham Lane Egham Surrey UK

Campus Cerro Largo Universidade Federal da Fronteira Sul Rua Jacob Haupenthal Cerro Largo RS CEP 158097900 000 Brazil

Center for Advanced Biotechnology and Medicine Rutgers University Robert Wood Johnson Medical School 679 Hoes Lane West Piscataway NJ 08854 USA

Center for Biodiversity and Global Change Yale University 165 Prospect St New Haven CT 06511 USA

Center for Biological Control Department of Zoology and Entomology Rhodes University 1 Lower University Road Barratt Complex Biological Sciences BuildingEastern Cape Makhanda South Africa

Center for Biological Data Science Virginia Commonwealth University 1000 W Cary St Box 842030 Richmond VA 23284 2030 USA

Center for Ecological Dynamics in a Novel Biosphere Department of Biology Aarhus University Ny Munkegade 114 116 8000 Aarhus C Denmark

Center for Ecosystem Science and Society Northern Arizona University PO Box 5620 Flagstaff AZ 86011 USA

Center for Impact UCLA Anderson University of California 110 Westwood Plaza Gold Hall Suite B 201L Los AngelesLos Angeles CA 90095 1481 USA

Center for Limnology University of Wisconsin Madison 680 N Park St Madison WI 53706 USA

Center for Watershed Sciences University of California Davis 1 Shields Ave Davis CA 95616 USA

Centre d'Études Biologiques de Chizé UMR 7372 Université de la Rochelle Centre National de la Recherche Scientifique 405 route de Prissé la Charrière 79360 Villiers en Bois France

Centre d'Investigations Clinique Plurithématique Institut Lorrain du Coeur et des Vaisseaux Université de Lorraine Inserm1433 CIC P CHRU de Nancy bâtiment Louis Mathieu 5 rue du Morvan 54500 Vandoeuvre les nancy France

Centre for Conservation Science RSPB 2 Lochside View Edinburgh EH12 9DH UK

Centre for Ecological Sciences Indian Institute of Science Indian Institute of Science Bengaluru Karnataka 560012 India

Centre for Ecology and Conservation University of Exeter Penryn Campus Penryn Cornwall TR10 9FE UK

Centre for Ecology Evolution and Environmental Changes and CHANGE Global Change and Sustainability Institute Faculdade de Ciências Universidade de Lisboa 1749 016 Lisbon Portugal

Chair of Meteorology Institute for Hydrology and Meteorology Faculty of Environmental Sciences Technische Universität Dresden Pienner Str 23 01737 Tharandt Germany

Chaire de recherche en intégrité écologique Université du Québec à Trois Rivières 3351 Boul Des Forges Trois Rivières QC G8Z 4M3 Canada

CNRS University of Rennes 263 Avenue du Général Leclerc 35042 Rennes France

College of Forestry Oregon State University 3100 SW Jefferson Way Corvallis OR 97333 USA

College of Natural Resources North Carolina State University Jordan Hall 2800 Faucette Dr Raleigh NC 27607 USA

Departamento de Biodiversidad Ecología y Evolución Universidad Complutense de Madrid C de José Antonio Novais 12 28040 Madrid Spain

Departamento de Ciencias Biológicas Universidad de los Andes Carrera 1 Nº 18A 12 111711 Bogotá Bogotá D C Colombia

Departamento de Ecologia e Zoologia Centro de Ciências Biológicas Universidade Federal de Santa Catarina UFSC Campus Universitário Córrego Grande Florianópolis SC; CEP Florianópolis 88040 900 Brazil

Département des Sciences biologiques Université du Québec à Montréal 141 Avenue du Président Kennedy Montréal Québec H2X 1Y4 Canada

Department of Animal Behaviour Bielefeld University Konsequenz 45 33615 Bielefeld Germany

Department of Animal Breeding and Genetics Swedish University of Agricultural Sciences Box 7023 750 07 Uppsala Sweden

Department of Animal Health and Production Oyo State College of Agriculture and Technology Igbo Ora 201103 Oyo State Nigeria

Department of Anthropology University of Michigan 1085 S University Ave Ann Arbor MI 48109 USA

Department of Aquatic Resources Swedish University of Agricultural Sciences Almas allé 5 756 51 Uppsala Sweden

Department of Arid Land Agriculture King Abdulaziz University Jeddah 80200 Kingdom of Saudi Arabia

Department of Behavioural Ecology Bielefeld University Konsequenz 45 33615 Bielefeld Germany

Department of Behavioural Neurobiology Max Planck Institute for Biological Intelligence Eberhard Gwinner Strasse 82319 Seewiesen Oberbayern Germany

Department of Biological and Environmental Science University of Jyväskylä Survontie 9C 40500 Jyväskylä Finland

Department of Biological Science University of Rhode Island 9 East Alumni Ave Kingston RI 02881 USA

Department of Biological Sciences Binghamton University 4400 Vestal Parkway East Binghamton NY 13902 USA

Department of Biological Sciences Boise State University 1910 W University Dr Boise ID 83725 USA

Department of Biological Sciences California State Polytechnic University Pomona USA

Department of Biological Sciences Macquarie University 205ACR Culloden Road Macquarie Park New South Wales 2113 Australia

Department of Biological Sciences Middle East Technical University Üniversiteler Mahallesi Dumlupınar Bulvarı No 1 06800 Çankaya Ankara Turkey

Department of Biological Sciences Purdue University 915 W State Street West Lafayette IN 47907 USA

Department of Biological Sciences San José State University 129 S 10th Street San Jose CA 95112 USA

Department of Biological Sciences University of Alberta Edmonton AB T6G 2R3 Canada

Department of Biological Sciences University of Arkansas 850 W Dickson Street SCEN601 Fayetteville AR 72701 USA

Department of Biological Sciences University of Bergen Postboks 7803 N 5020 Bergen Norway

Department of Biological Sciences University of Southern Maine 70 Falmouth St Portland ME 04103 USA

Department of Biology Biology Research and Administration Building University of Oxford 11a Mansfield Rd Oxford OX1 3SZ UK

Department of Biology Brigham Young University Brigham Young University Brigham Young University Provo UT 84602 USA

Department of Biology McMaster University 1280 Main Street West Hamilton ON L8S 4K1 Canada

Department of Biology Memorial University of Newfoundland 45 Arctic Ave St John's NL A1C5S7 Canada

Department of Biology Missouri State University 910 S John Q Hammons Pkwy Springfield MO 65897 USA

Department of Biology Rhodes College 2000 N Parkway Memphis TN 38112 USA

Department of Biology St Norbert College 100 Grant St De Pere WI 54115 USA

Department of Biology University of Antwerp Universiteitsplein 1 2610 Wilrijk Belgium

Department of Biology University of Massachusetts 1 Campus Center Way Amherst MA 01003 USA

Department of Biology University of Regina 3737 Wascana Pkwy Regina Saskatchewan S4S 0A2 Canada

Department of Biology University of Turku Turun Yliopisto FI 20014 Turku Finland

Department of Biology Whitman College 345 Boyer Ave Walla Walla WA 99362 USA

Department of Biology Wilfrid Laurier University 75 University Ave West Waterloo Ontario N2L 3C5 Canada

Department of Botany University of Wyoming Laramie WY 82071 USA

Department of Cell and Developmental Biology Division of Biosciences University College London London UK

Department of Clinical Neuroscience Karolinska Institutet Nobels väg 9 171 77 Stockholm Sweden

Department of Computational Landscape Ecology Helmholtz Centre for Environmental Research UFZ Permoserstraße 15 04318 Leipzig Germany

Department of Earth and Environmental Sciences KU Leuven Celestijnenlaan 200E 3001 Leuven Belgium

Department of Earth Sciences Montana State University Bozeman MT 59717 USA

Department of Ecology and Evolutionary Biology Biological Science Building University of Michigan 1105 North University Avenue Ann Arbor MI 48109 1085 USA

Department of Ecology and Evolutionary Biology Cornell University 215 Tower Road Ithaca NY 14853 USA

Department of Ecology and Evolutionary Biology School of Biological Sciences University of California 321 Steinhaus Hall IrvineIrvine CA 92697 USA

Department of Ecology and Evolutionary Biology University of Connecticut 75 N Eagleville Rd Storrs CT 06226 USA

Department of Ecology and Evolutionary Biology University of Toronto 25 Willcocks St Toronto ON M5S 3B2 Canada

Department of Ecology and global change Centro de Investigaciones sobre Desertificación Consejo Superior de Investigaciones Cientificas Spain

Department of Ecology Behavior and Evolution University of California San Diego 9500 Gilman Dr La Jolla CA 92093 USA

Department of Ecology Evolution and Behavior University of Minnesota Ecology Building 1987 Upper Buford Cir St PaulSt Paul MN 55108 USA

Department of Ecology Evolution and Organismal Biology Iowa State University 2200 Osborn Dr Ames IA 50011 USA

Department of Ecology Swedish University of Agricultural Sciences Ulls Väg 16 750 07 Uppsala Sweden

Department of Entomology The Ohio State University 1680 Madison Ave Wooster OH 44691 USA

Department of Environmental Sciences Western Norway University of Applied Sciences P O box 133 6851 Sogndal Norway

Department of Evolution Ecology and Organismal Biology University of California 900 University Ave RiversideRiverside CA 92521 USA

Department of Evolutionary Anthropology Duke University 130 Science Dr Durham NC 27708 USA

Department of Evolutionary Anthropology University of Vienna Djerassiplatz 1 1030 Wien Austria

Department of Evolutionary Biology and Environmental Studies University of Zurich Winterthurerstrasse 190 8057 Zürich Switzerland

Department of Evolutionary Biology Bielefeld University North Rhine Westphalia Konsequenz 45 33615 Bielefeld Germany

Department of Fisheries Wildlife and Conservation Biology University of Minnesota Twin Cities 135 Skok Hall 2003 Upper Buford Circle St Paul MN 55108 USA

Department of Food Science and Technology Virginia Polytechnic Institute and State University 22 Food Science Building 360 Duck Pond Drive Virginia Tech Blacksburg VA 24061 USA

Department of Forest and Wildlife Ecology University of Wisconsin Madison 1630 Linden Drive Madison WI 53706 USA

Department of Forest Ecosystems and Society College of Forestry Oregon State University Corvallis OR 97333 USA

Department of Forest Ecosystems and Society Oregon State University 321 Richardson Hall Corvallis OR 97331 USA

Department of Geography McGill University 805 Sherbrooke Street West Montreal Quebec H3A 0B9 Canada

Department of Integrative Biology The University of Texas at Austin 2415 Speedway C0930 Austin Texas 78712 USA

Department of Integrative Biology University of California Valley Life Science Building 5075 BerkeleyBerkeley CA 94720 USA

Department of Integrative Biology University of Colorado P O Box 173364 DenverDenver CO 80217 3364 USA

Department of Integrative Biology University of Guelph 50 Stone Rd E Guelph ON N1G 2W1 Canada

Department of Integrative Biology Valley Life Sciences Building 3140 University of California University of California Berkeley BerkeleyBerkeley CA 94720 USA

Department of Life Sciences Aberystwyth University Penglais Aberystwyth SY23 3DA UK

Department of Life Sciences Ben Gurion University of the Negev P O Box 653 84105 Beer Sheva Israel

Department of Life Sciences Imperial College London Buckhurst road Berkshire SL5 7PY UK

Department of Marine Sciences University of Gothenburg Box 461 SE 40530 Gothenburg Sweden

Department of Natural History Norwegian University of Science and Technology Høgskoleringen 1 7034 Trondheim Norway

Department of Natural Resource Sciences McGill University 21111 Lakeshore Rd Ste Anne de Bellevue Montreal QC H9X 3V9 Canada

Department of Natural Resources Newe Ya'ar Research Center Agricultural Research Organization POB 1021 3009500 Ramat Yishay Israel

Department of Natural Sciences Baruch College City University of New York 17 Lexington Avenue New York NY 10010 USA

Department of Ornithology Academy of Natural Sciences of Drexel University 1900 Benjamin Franklin Parkway Philadelphia PA 19096 USA

Department of Ornithology Max Planck Institute for Biological Intelligence Eberhard Gwinner Str 7 82319 Seewiesen Germany

Department of Plant Ecology University of Hohenheim Institute of Landscape and Plant Ecology Ottilie Zeller Weg 70599 Stuttgart Germany

Department of Plant Protection Faculty of Agriculture Department of Plant Protection Faculty of Agriculture Ordu University Ordu University 52200 Altinordu Ordu Turkey

Department of Psychiatry Yale School of Medicine Yale University 389 Whitney Ave New Haven CT 06511 USA

Department of Psychology and Vision Science University of Birmingham 52 Pritchatts Road Edgbaston Baily Thomas GrantBirmingham B15 2TT UK

Department of Psychology The University of Edinburgh 7 George Square Edinburgh EH9 1HB UK

Department of Psychology University of Bath 10 West Bath BA2 7AY UK

Department of Science and Environment Lake Superior State University 650 W Easterday Ave Sault Sainte Marie MI 49783 USA

Department of Statistics University of Auckland Auckland New Zealand

Department of Wildlife Fish and Conservation Biology University of California 1 Shields Ave DavisDavis CA 95616 USA

Department of Wildlife Fish and Environmental Studies Swedish University of Agricultural Sciences Skogsmarksgränd 17 SE 907 36 Umeå Sweden

Department of Wildlife Fisheries and Conservation Biology University of Maine 5755 Nutting Hall Room 210 Orono ME 04469 5755 USA

Department of Zoology University of Cambridge Downing St Cambridge CB2 3EJ UK

Dept Ecologia Instituto de Biologia Universidade Federal do Rio de Janeiro Av Carlos Chagas Filho 373 Rio de Janeiro RJ CP 6820021942 902 Brazil

Dept of Integrative Biology University of Texas at Austin 2415 Speedway C0930 Austin TX USA

Dipartimento di Biologia Università di Roma Tor Vergata Via Cracovia 1 00133 Rome Italy

Earth and Environmental Sciences Wesleyan University 45 Wyllys Ave Middletown CT 06459 USA

Earth and Life Institute Ecology and Biodiversity UCLouvain Croix du Sud 4 L7 07 04 1348 Louvain la Neuve Belgium

Ecology and Environmental Science Umeå University Linnaeus väg 6 907 36 Umeå Sweden

Ecology and Evolutionary Biology University of Arizona 1041 E Lowell St Tucson AZ 85721 USA

Ecology and Evolutionary Biology University of Tennessee Knoxville 569 Dabney Hall Knoxville TN 37996 USA

Ecology Department Universidade Federal de Goiás Av Esperança Campus Samambaia Goiânia Goiás 74690 900 Brazil

Ecology Evolution and Behavior Program Michigan State University East Lansing MI 48824 USA

Ecosystems and Global Change Group School of the Environment Trent University 1600 West Bank Road Peterborough Ontario K0L 2V0 Canada

Entomology and Nematology University of Florida 700 Experiment Station Rd Lake Alfred FL 33850 USA

Environmental Studies Elon University McMichael Science Building 2625 Campus Box Elon NC 27244 USA

Environmental Studies Wofford College 429 N Church St Spartanburg SC 29303 USA

Epidemiology and Surveillance Support Unit University of Lyon French Agency for Food Environmental and Occupational Health and Safety 31 Avenue Tony Garnier 69007 Lyon France

European Forest Institute Platz d Vereinten Nationen 7 53113 Bonn Germany

Evolution and Ecology Research Centre School of Biological Earth and Environmental Sciences University of New South Wales UNSW Sydney High Street 2052 Kensington NSW Australia

Evolutionary biology department Bielefeld University Bielefeld University Konsequenz 45 33615 Bielefeld Germany

Facultad de Ciencias Universidad de la República Iguá 4225 11400 Montevideo Montevideo Uruguay

Faculty of Environmental Sciences Czech University of Life Sciences Prague Czech Republic Kamýcká 129 Praha Suchdol 165 00 Czech Republic

Faculty of Life Sciences Bar Ilan University Ramat Gan 529000 Israel

Fakultät für Biologie Arbeitsgruppe Evolutionsbiologie Universität Bielefeld Morgenbreede 45 33615 Bielefeld Germany

FitzPatrick Institute of African Ornithology University of Cape Town University of Cape Town Private Bag X3 Rondebosch Cape Town 7701 South Africa

Forest and Rangeland Stewardship Colorado State University 1472 Campus Delivery Fort Collins CO 80523 1472 USA

Forestry and Environmental Conservation National Bobwhite and Grassland Initiative Clemson University 243 Lehotsky Hall Clemson SC 29634 USA

Fram Project AS Ymers vei 2 0588 Oslo Norway

Future Regions Research Centre Federation University Australia Mt Helen VIC 3350 Australia

Graduate School of Biomedical Sciences Tufts University 136 Harrison Ave 813 Boston MA 02111 USA

Grand Bay National Estuarine Research Reserve 6005 Bayou Heron Rd Moss Point MS 39562 USA

Groningen Institute for Evolutionary Life Sciences University of Groningen Nijenborgh 7 9747 AG Groningen Netherlands

Harvard Forest Harvard University 324 N Main St Petersham MA 01366 USA

Helmholtz AI Helmholtz Zentrum Muenchen Ingolstaedter Landstr 1 85764 Neuherberg Germany

Hiram College 11700 Dean St Biology Hiram OH 44234 USA

IFM Biology Linköping University 581 83 Linköping Sweden

Institut de biologie Université de Neuchâtel Emile Argand 11 2000 Neuchâtel Switzerland

Institut de recherche en biologie végétale Université de Montréal 4101 Sherbrooke St E Montréal Québec H1X 2B2 Canada

Institut de recherche sur les forêts Université du Québec en Abitibi Témiscamingue 445 Boulevard de l'Université Rouyn Noranda QC J9X 5E4 Canada

Institute for Biodiversity Animal Health and Comparative Medicine University of Glasgow Glasgow G12 8QQ UK

Institute for Environmental Sciences VU Amsterdam De Boelelaan 1111 1081 HV Amsterdam The Netherlands

Institute for Evolution and Biodiversity University of Muenster Huefferstr 1 DE 48149 Muenster Germany

Institute for Interdisciplinary Mountain Research OeAW GLORIA Silbergasse 30 3 A 1190 Wien Austria

Institute of Avian Research Vogelwarte Helgoland An der Vogelwarte 21 26386 Wilhelmshaven Germany

Institute of Ecology and Environmental Sciences Univ Paris Est Creteil 61 avenue du Général de Gaulle 94010 Créteil France

Institute of Ecology and Evolution University of Edinburgh The University of Edinburgh King's Buildings Charlotte Auerbach Road Edinburgh EH9 3FL UK

Institute of Environment and Department of Biological Sciences Florida International University 3000 NE 151st St North Miami FL 33181 USA

Institute of Environmental Sciences Jagiellonian University Gronostajowa 7 30 387 Krakow Poland

Institute of Evolutionary Biology University of Edinburgh King's Buildings Edinburgh EH9 3JW UK

Institute of Evolutionary Ecology and Conservation Genomics Ulm University Albert Einstein Allee 11 89081 Ulm Germany

Institute of Evolutionary Sciences Montpellier University of Montpellier CNRS IRD Montpellier France

Institute of Plant Sciences University of Bern Altenbergrain 21 3013 Bern Switzerland

Institute of the Environment Florida International University 3000 NE 151st St North Miami FL 33181 USA

Institute of Zoology Technische Universität Dresden Zellescher Weg 20b 01217 Dresden Germany

Instituto de Ciencia Animal Facultad de Ciencias Veterinarias Universidad Austral de Chile Campus Isla Teja s n Valdivia Chile

Instituto Universitario de Investigación en Gestión Forestal Sostenible Universidad de Valladolid Av Madrid 44 34071 Palencia Spain

International Institute for Applied Systems Analysis Schlossplatz 1 A 2361 Laxenburg Austria

ISEM University of Montpellier CNRS Place Eugène BataillonCedex 05 34095 Montpellier France

Konrad Lorenz Institute for Ethology University of Veterinary Medicine Savoyenstrasse 1A 1160 Vienna Austria

Laboratoire d'Ethologie Expérimentale et Comparée LEEC Université Sorbonne Paris Nord 99 avenue Jean Baptiste Clément UR444393430 Villetaneuse France

Laboratorio de Invasiones Biológicas Instituto de Ecología y Biodiversidad Victoria 631 Concepción Chile

Länsstyrelsen Östergötland Östgötagatan 3 58186 Linköping Sweden

LLC Catbird Stats PO Box 2018 Gautier MS 39553 USA

Lothian Analytical Services Public Health Scotland 1 South Gyle Crescent Edinburgh EH12 9EB UK

Marine and Continental Waters IRTA Carretera Poble Nou Km 5 5 43540 La Ràpita Catalonia Spain

Mention Zoologie et Biodiversité Animale Faculté des Sciences Université d'Antananarivo Mention Zoologie et Biodiversié Animale Université d'Antananarivo BP 906 101 Antananarivo Madagascar

Mississippi Based RESTORE Act Center of Excellence University of Southern Mississippi 703 E Beach Drive Ocean Springs MS 39564 USA

Molecular Ecology Group 28922 Corso Tonolli 50 Verbania Italy

MRC Biostatistics Unit University of Cambridge East Forvie Building Forvie Site Robinson Way Cambridge CB2 0SR UK

Natural Resource Ecology and Management Iowa State University 2310 Pammel Dr Ames IA 50011 USA

New South Wales Department of Primary Industries Fisheries Locked Bag 1 Nelson Bay NSW 2315 Australia

Nomad Ecology 822 Main Street Martinez CA 94553 USA

Office for National Statistics Segensworth Rd Titchfield Fareham PO15 5RR UK

Organismal and Evolutionary Biology Research Programme Faculty of Biological and Environmental Sciences University of Helsinki 00014 Helsinki Finland

Pastures Systems and Watershed Management Research Unit USDA Agricultural Research Service USDA ARS PSWMRU Bldg 3702 Curtin Road University Park PA 16802 USA

Phnom Penh Cambodia

Plant Sciences University of California 1 Shields Ave DavisDavis CA 95616 USA

Programa de Pós Graduação em Ecologia Instituto de Biologia Centro de Ciências da Saúde Universidade Federal do Rio de Janeiro Cidade Universitária Av Carlos Chagas Filho 373 Rio de Janeiro RJ CEP 21941 902 Brazil

Research Data Management Leibniz Centre for Agricultural Landscape Research Eberswalder Straße 84 15374 Müncheberg Germany

Salisbury University 1101 Camden Ave Biology Salisbury MD 21801 USA

School of Agriculture and Environmental Science University of Southern Queensland 487 535 West Street Toowoomba Qld 4350 Australia

School of Agriculture Food and Ecosystem Sciences The University of Melbourne Grattan Street Parkville Victoria 3010 Australia

School of Agriculture Food and Ecosystem Sciences University of Melbourne Grattan Street Parkville Victoria 3010 Australia

School of Applied Sciences School of Applied Sciences University of Brighton University of Brighton Lewes Road Brighton BN2 4GJ UK

School of Biodiversity One Health and Veterinary Medicine University of Glasgow University Avenue Glasgow G12 8QQ UK

School of Biological and Environmental Sciences Liverpool John Moores University James Parsons Building Byrom Street Liverpool L3 3AF UK

School of Biological and Forensic Sciences University of South Wales The Alfred Russel Wallace Building 9 Graig Fach Glyntaff Pontypridd CF37 4BB UK

School of Biological and Marine Sciences University of Plymouth Drake Circus Plymouth Devon PL4 8AA UK

School of Biological Earth and Environmental Sciences University of New South Wales Randwick Sydney NSW 2052 Australia

School of Biological Earth and Environmental Sciences University of New South Wales Sydney NSW 2052 Australia

School of Biological Sciences Monash University Rainforest Walk 25 Clayton Victoria Australia

School of Biological Sciences The University of Hong Kong Pokfulam Road Hong Kong SAR China

School of Biological Sciences University of Aberdeen King Street Aberdeen AB244FX UK

School of Biological Sciences University of Bristol 24 Tyndall Avenue Bristol BS8 1TQ UK

School of Biological Sciences University of Western Australia 35 Stirling Highway Crawley Western Australia 6009 Australia

School of Biology and Environmental Science University College Dublin Dublin 4 Belfield D04 V1W8 Ireland

School of Forestry Northern Arizona University 200 E Pine Knoll Dr Flagstaff AZ 86001 USA

School of Historical and Philosophical Studies University of Melbourne Grattan Street Parkville Victoria 3010 Australia

School of Information University of Arizona 1103 E 2nd St Tucson AZ 85721 USA

School of Integrative Biological and Chemical Sciences The University of Texas Rio Grande Valley One West University Boulevard Brownsville TX 78520 USA

School of Life and Environmental Sciences Deakin University 221 Burwood Highway Burwood VIC 3125 Australia

School of Life and Environmental Sciences Deakin University Geelong Victoria Australia

School of Life and Environmental Sciences The University of Sydney Camperdown NSW 2006 Australia

School of Mathematics and Statistics and Centre for Research in Ecological and Environmental Modelling University of St Andrews Buchanan Gardens St Andrews Scotland KY16 9LZ UK

School of Mathematics and Statistics University of Melbourne Parkville VIC 3052 Australia

School of Molecular Biosciences College of Medical Veterinary and Life Sciences University of Glasgow Glasgow G12 8Qq UK

School of Natural Sciences Macquarie University Balaclava Rd Macquarie Park Sydney NSW 2109 Australia

School of Natural Sciences University of Tasmania TAS Private Bag 55 Hobart 7001 Australia

School of Psychology and Social Work University of Hull Cottingham Rd Hull HU6 7RX UK

School of Public Health and Preventive Medicine Monash University 750 Collins Street Docklands VIC 3008 Australia

School of Sciences Center for Health and Cognition Bath Spa University Newton Park Bath BA2 9BN UK

School of the Environment Faculty of Science The University of Queensland The University of Queensland Brisbane QLD 4072 Australia

School of the Environment Yale University 195 Prospect Street New Haven CT 06511 USA

Sección Ornitología Universidad Nacional de La Plata Paseo del Bosqur s n La Plata B1900FWA Buenos Aires Argentina

South Iceland Research Centre University of Iceland Lindarbraut 4 840 Laugarvatn Iceland

Southern Research Station USDA Forest Service PO Box 700 New Ellenton SC 29809 USA

Statistical Ecotoxicology Bayreuth Center of Ecology and Environmental Research University of Bayreuth Universitätsstraße 30 95440 Bayreuth Germany

STEM Center Southern Illinois University Edwardsville 1 Hairpin Dr Edwardsville IL 62026 USA

Technology and Society Department Rochester Institute of Technology 7 Lomb Memorial Drive Rochester NY 14623 USA

The Nature Conservancy 258 Main Street Lander WY 82520 USA

UMR 1224 ECOBIOP Université de Pau et des Pays de l'Adour 173 Route de Saint Jean de Luz 64310 Saint Pée sur Nivelle France

UMR 1309 ASTRE CIRAD Campus international de Baillarguet 34398 Montpellier France

United States Department of Agriculture Agricultural Research Service 1701 10th Ave SW Mandan ND 58554 USA

Universidad de los Andes Carrera 1 18A 12 Bogotá Colombia

Université de Montpellier ISEM University of Montpellier CNRS EPHE 34000 Montpellier IRD France

Université du Québec à Trois Rivières 3351 boulevard des Forges Trois Rivières G8Z 4M3 Canada

University of Antwerp Universiteitsplein 1 2610 Wilrijk België Belgium

University of Cambridge Trinity Ln The Old Schools Cambridge CB2 1TN UK

University of Guelph 50 Stone Road East Guelph Ontario N1G 2W1 Canada

University of Wisconsin 1525 Observatory Dr Madison Madison WI 53706 USA

Vive Crop Protection 6275 Northam Drive Suite 1 Mississauga ON L4V 1Y8 Canada

Wake Forest University 1834 Wake Forest Road Winston Salem NC 27109 USA

Washington USA

Whitebark Institute 3399 Main Street Suite W5 Mammoth Lakes CA 93546 USA

Whitney Laboratory for Marine Bioscience University of Florida 9505 N Ocean Shore Blvd St Augustine Gainesville FL 32080 USA

Wildland Resources Department Utah State University 5200 Old Main Hill Logan UT 84322 USA

Zobrazit více v PubMed

Aczel, Balazs, Barnabas Szaszi, Gustav Nilsonne, Olmo R van den Akker, Casper J Albers, Marcel ALM van Assen, Jojanneke A Bastiaansen, et al. 2021. “Consensus-Based Guidance for Conducting and Reporting Multi-Analyst Studies. eLife. 10 (November). 10.7554/elife.72185. PubMed PMC

Allaire, J. J., Charles Teague, Carlos Scheidegger, Yihui Xie, and Christophe Dervieux. 2024. Quarto. 10.5281/zenodo.5960048.

Allaire, JJ, Yihui Xie, Christophe Dervieux, Jonathan McPherson, Javier Luraschi, Kevin Ushey, Aron Atkins, et al. 2024. rmarkdown: Dynamic Documents for r. https://github.com/rstudio/rmarkdown.

Arif S, Aaron MacNeil M. Predictive models aren’t for causal inference. Ecology Letters. 2022;25(8):1741–5. 10.1111/ele.14033. PubMed

Arif, Suchinta, and M. Aaron MacNeil. 2023. “Applying the Structural Causal Model Framework for Observational Causal Inference in Ecology.” Ecological Monographs. 93(1):e1554. 10.1002/ecm.1554.

Arnold, Jeffrey B. 2024. ggthemes: Extra Themes, Scales and Geoms for “ggplot2”. https://jrnold.github.io/ggthemes/.

Atkinson, Joe, Lars A. Brudvig, Max Mallen-Cooper, Shinichi Nakagawa, Angela T. Moles, and Stephen P. Bonser. 2022. “Terrestrial Ecosystem Restoration Increases Biodiversity and Reduces Its Variability, but Not to Reference Levels: A Global Meta-Analysis. Ecology Letters. 25 (7): 1725–37. 10.1111/ele.14025. PubMed PMC

Auspurg Katrin, Brüderl Josef. Has the Credibility of the Social Sciences Been Credibly Destroyed? Reanalyzing the ‘Many Analysts, One Data Set’ Project. Socius. 2021;7:23780231211024420. 10.1177/23780231211024421.

Bartoń, Kamil. MuMIn: Multi-Model Inference. R package version 1.47.5. 2023. https://cran.r-project.org/web/packages/MuMIn.

Baselga, Andres, David Orme, Sebastien Villeger, Julien De Bortoli, Fabien Leprieur, Maxime Logez, Sara Martinez-Santalla, et al. 2023. betapart: Partitioning Beta Diversity into Turnover and Nestedness Components. https://CRAN.R-project.org/package=betapart.

Bates, Douglas, Martin Mächler, Ben Bolker, and Steve Walker. 2015. “Fitting Linear Mixed-Effects Models Using lme4. 2015;67(1):48. 10.18637/jss.v067.i01.

Blake, Kevin. 2022. NatParksPalettes: Color Palettes Inspired by National Parks. https://github.com/kevinsblake/NatParksPalettes.

Bolker, Ben, David Robinson, Dieter Menne, Jonah Gabry, Paul Buerkner, Chrisopher Hau, William Petry, et al. 2024. broom.mixed: Tidying Methods for Mixed Models. https://github.com/bbolker/broom.mixed.

Borenstein Michael, Higgins Julian P. T, Hedges Larry, Rothstein Hannah. Basics of Meta-Analysis: I2 Is Not an Absolute Measure of Heterogeneity. Research Synthesis Methods. 2017;8:5–18. 10.1002/jrsm.1230. PubMed

Botvinik-Nezer Rotem, Holzmeister Felix, Camerer Colin F, Dreber Anna, Huber Juergen, Johannesson Magnus, Kirchler Michael, et al. Variability in the Analysis of a Single Neuroimaging Dataset by Many Teams. Nature. 2020;582(7810):84–8. PubMed PMC

Breznau Nate, Rinke Eike Mark, Wuttke Alexander, Nguyen Hung H. V, Adem Muna, Adriaans Jule, Alvarez-Benjumea Amalia, et al. Observing Many Researchers Using the Same Data and Hypothesis Reveals a Hidden Universe of Uncertainty. Proceedings of the National Academy of Sciences. 2022;119(44): e2203150119. 10.1073/pnas.2203150119. PubMed PMC

Briga, Michael, and Simon Verhulst. 2021. “Mosaic Metabolic Ageing: Basal and Standard Metabolic Rates Age in Opposite Directions and Independent of Environmental Quality, Sex and Life Span in a Passerine. Functional Ecology. 35 (5): 1055–68. 10.1111/1365-2435.13785.

Brooks, Mollie E., Kasper Kristensen, Koen J. van Benthem, Arni Magnusson, Casper W. Berg, Anders Nielsen, Hans J. Skaug, Martin Maechler, and Benjamin M. Bolker. 2017. “glmmTMB Balances Speed and Flexibility Among Packages for Zero-Inflated Generalized Linear Mixed Modeling. The R Journal. 9(2):378–400. 10.32614/RJ-2017-066.

Buck Robert J, Fieberg John, Larkin Daniel J. The use of weighted averages of Hedges’ d in meta-analysis: Is it worth it? Methods in Ecology and Evolution. 2022;13(5):1093–105. 10.1111/2041-210X.13818.

Burnham, K. P., and D. R. Anderson. 2002. Model Selection and Multimodel Inference: A Practical Information-Theoretical Approach. Book. 2nd ed. New York: Springer-Verlag. 10.1007/b97636.

Cade, Brian S. 2015. “Model Averaging and Muddled Multimodel Inferences. Ecology. 96(9):2370–82. http://www.jstor.org.ezproxy.whitman.edu/stable/24702343. PubMed

Capilla-Lasheras Pablo, Thompson Megan J, Sánchez-Tójar Alfredo, Haddou Yacob, Branston Claire J, Réale Denis, Charmantier Anne, Dominoni Davide M. A Global Meta-Analysis Reveals Higher Variation in Breeding Phenology in Urban Birds Than in Their Non-Urban Neighbours. Ecology Letters. 2022;25(11):2552–70. 10.1111/ele.14099. PubMed PMC

Coretta Stefano, Casillas Joseph V, Roessig Simon, Franke Michael, Ahn Byron, Al-Hoorie Ali H, Al-Tamimi Jalal, et al. Multidimensional Signals and Analytic Flexibility: Estimating Degrees of Freedom in Human-Speech Analyses. Advances in Methods and Practices in Psychological Science. 2023;6(3):25152459231162570. 10.1177/25152459231162567.

Dancho, Matt, and Davis Vaughan. 2023. Timetk: A Tool Kit for Working with Time Series. https://CRAN.R-project.org/package=timetk.

DeKogel, C. H. 1997. “Long-Term Effects of Brood Size Manipulation on Morphological Development and Sex-Specific Mortality of Offspring. Journal of Animal Ecology. 66(2):167–78. <Go to ISI>://WOS:A1997WQ19600003.

Deressa, Teshome, David Stern, Jaco Vangronsveld, Jan Minx, Sebastien Lizin, Robert Malina, and Stephan Bruns. 2023. “More Than Half of Statistically Significant Research Findings in the Environmental Sciences Are Actually Not. EcoEvoRxiv. 10.32942/X24G6Z.

Dormann, Carsten F., Jane Elith, Sven Bacher, Carsten Buchmann, Gudrun Carl, Gabriel Carré, Jaime R. García Marquéz, et al. 2013. “Collinearity: A Review of Methods to Deal with It and a Simulation Study Evaluating Their Performance. Ecography. 36(1):27–46. 10.1111/j.1600-0587.2012.07348.x.

Fanelli Daniele, Costas Rodrigo, Ioannidis John P. A. Meta-Assessment of Bias in Science. Proceedings of the National Academy of Sciences. 2017;114:3714–9. 10.1073/pnas.1618569114. PubMed PMC

Fanelli Daniele, Ioannidis John P. A. US Studies May Overestimate Effect Sizes in Softer Research. Proceedings of the National Academy of Sciences. 2013;110(37):15031–6. 10.1073/pnas.1302997110. PubMed PMC

Fidler Fiona, Burgman Mark A, Cumming Geoff, Buttrose Robert, Thomason Neil. Impact of Criticism of Null-Hypothesis Significance Testing on Statistical Reporting Practices in Conservation Biology. Conservation Biology. 2006;20(5):1539–44. 10.1111/j.1523-1739.2006.00525.x. PubMed

Fidler Fiona, Chee Yung En, Wintle Bonnie C, Burgman Mark A, McCarthy Michael A, Gordon Ascelin. Metaresearch for Evaluating Reproducibility in Ecology and Evolution. BioScience. 2017;67(3):282–9. 10.1093/biosci/biw159. PubMed PMC

Firke, Sam. 2023. janitor: Simple Tools for Examining and Cleaning Dirty Data. https://github.com/sfirke/janitor.

Forstmeier Wolfgang, Wagenmakers Eric-Jan, Parker TH. Detecting and Avoiding Likely False-Positive Findings – a Practical Guide. Biological Reviews. 2017;92:1941–68. 10.1111/brv.12315. PubMed

Fraser Hannah, Parker Tim, Nakagawa Shinichi, Barnett Ashley, Fidler Fiona. Questionable Research Practices in Ecology and Evolution. PLOS ONE. 2018;13(7): e0200303. 10.1371/journal.pone.0200303. PubMed PMC

Gamer, Matthias, Jim Lemon, and Ian Fellows Puspendra Singh. 2019. irr: Various Coefficients of Interrater Reliability and Agreement. https://www.r-project.org.

Gelman Andrew, Loken Eric. The Garden of Forking Paths: Why Multiple Comparisons Can Be a Problem, Even When There Is No ‘Fishing Expedition’ or ‘p-Hacking’ and the Research Hypothesis Was Posited Ahead of Time. Department of Statistics: Columbia University; 2013.

Gelman Andrew, Weakliem David. Of Beauty, Sex, and Power. American Scientist. 2009;97:310–6.

Gould Elliot, Fraser Hannah S, Nakagawa Shinichi, Parker Timothy H. ManyEcoEvo: Meta-Analyse Data from ManyAnalyst Style Studies. 2023. Zenodo. 10.5281/zenodo.10046153.

Gould Ellliot, Fraser Hannah S, Nakagawa Shinichi, Parker Timothy H. egouldo/ManyAnalysts: Manuscript Source Code for “Same data, different analysts: variation in effect sizes due to analytical decisions in ecology and evolutionary biology.” 2024. Zenodo. 10.5281/zenodo.13850927.Version2.0.2. PubMed PMC

Grueber, C. E., S. Nakagawa, R. J. Laws, and I. G. Jamieson. 2011. “Multimodel Inference in Ecology and Evolution: Challenges and Solutions. Journal of Evolutionary Biology. 24(4):699–711. 10.1111/j.1420-9101.2010.02210.x. PubMed

Harrell Jr, Frank E. 2024. Hmisc: Harrell Miscellaneous. https://hbiostat.org/R/Hmisc/.

Hester, Jim, Lionel Henry, Kirill Müller, Kevin Ushey, Hadley Wickham, and Winston Chang. 2024. withr: Run Code “With” Temporarily Modified Global State. https://withr.r-lib.org.

Higgins Julian P T, Thompson Simon G, Deeks Jonathan J, Altman Douglas G. Measuring Inconsistency in Meta-Analyses. BMJ. 2003;327(7414):557–60. 10.1136/bmj.327.7414.557. PubMed PMC

Huntington-Klein, Nick, Andreu Arenas, Emily Beam, Marco Bertoni, Jeffrey R. Bloem, Pralhad Burli, Naibin Chen, et al. 2021. “The Influence of Hidden Researcher Decisions in Applied Microeconomics. Economic Inquiry. 59(3):944–60. 10.1111/ecin.12992.

Iannone, Richard, Joe Cheng, Barret Schloerke, Ellis Hughes, Alexandra Lauer, and JooYoung Seo. 2024. gt: Easily Create Presentation-Ready Display Tables. https://gt.rstudio.com.

Jennions, M. D., C. J. Lortie, M. S. Rosenberg, and H. R. Rothstein. 2013. “Publication and Related Biases.” Book Section. In Handbook of Meta-Analysis in Ecology and Evolution, edited by J. Koricheva, J. Gurevitch, and K. Mengersen, 207–36. Princeton, USA: Princeton University Press.

Kassambara, Alboukadel. 2023. ggpubr: “ggplot2” Based Publication Ready Plots. https://rpkgs.datanovia.com/ggpubr/.

Kimmel Kaitlin, Avolio Meghan L, Ferraro Paul J. Empirical Evidence of Widespread Exaggeration Bias and Selective Reporting in Ecology. Nature Ecology & Evolution. 2023. 10.1038/s41559-023-02144-3. PubMed

Klein Richard A, Ratliff Kate A, Vianello Michelangelo, Adams Jr Reginald B, Bahník Štěpán, Bernstein Michael J, Bocian Konrad, et al. Investigating Variation in Replicability: A “Many Labs” Replication Project. Social Psychology. 2014;45(3):142–52. 10.1027/1864-9335/a000178.

Klein Richard A, Vianello Michelangelo, Hasselman Fred, Adams Byron G, Adams Reginald B, Alper Sinan, Aveyard Mark, et al. Many Labs 2: Investigating Variation in Replicability Across Samples and Settings. Advances in Methods and Practices in Psychological Science. 2018;1(4):443–90. 10.1177/2515245918810225.

Knight K. Mathematical Statistics. Book. New York: Chapman; Hall; 2000.

Koricheva, Julia, and Jessica Gurevitch. 2014. “Uses and Misuses of Meta-Analysis in Plant Ecology. Journal of Ecology. 102(4):828–44. 10.1111/1365-2745.12224.

Kou-Giesbrecht, Sian, and Duncan N. L. Menge. 2021. “Nitrogen-Fixing Trees Increase Soil Nitrous Oxide Emissions: A Meta-Analysis. Ecology. 102(8):e03415. 10.1002/ecy.3415. PubMed

Kuhn, Max, and Hannah Frick. 2022. multilevelmod: Model Wrappers for Multi-Level Models. https://github.com/tidymodels/multilevelmod.

Kuhn, Max, and Hadley Wickham. 2020. Tidymodels: A Collection of Packages for Modeling and Machine Learning Using Tidyverse Principles. https://www.tidymodels.org.

Kuznetsova, Alexandra, Per B. Brockhoff, and Rune H. B. Christensen. 2017. “lmerTest Package: Tests in Linear Mixed Effects Models. Journal of Statistical Software. 82(13):1–26. 10.18637/jss.v082.i13.

Landau, William Michael. 2021. “The Targets r Package: A Dynamic Make-Like Function-Oriented Pipeline Toolkit for Reproducibility and High-Performance Computing. Journal of Open Source Software. 6(57):2959. 10.21105/joss.02959.

Leybourne, Daniel J., Katharine F. Preedy, Tracy A. Valentine, Jorunn I. B. Bos, and Alison J. Karley. 2021. “Drought Has Negative Consequences on Aphid Fitness and Plant Vigor: Insights from a Meta-Analysis. Ecology and Evolution. 11(17):11915–29. 10.1002/ece3.7957. PubMed PMC

Lu, Xun, and Halbert White. 2014. “Robustness Checks and Robustness Tests in Applied Economics. Journal of Econometrics. 178:194–206. 10.1016/j.jeconom.2013.08.016.

Lüdecke, Daniel, Mattan S. Ben-Shachar, Indrajeet Patil, and Dominique Makowski. 2020. “Extracting, Computing and Exploring the Parameters of Statistical Models Using R. Journal of Open Source Software. 5(53):2445. 10.21105/joss.02445.

Lüdecke, Daniel, Mattan S. Ben-Shachar, Indrajeet Patil, Philip Waggoner, and Dominique Makowski. 2021. “performance: An R Package for Assessment, Comparison and Testing of Statistical Models. Journal of Open Source Software. 6(60):3139. 10.21105/joss.03139.

Lüdecke, Daniel, Indrajeet Patil, Mattan S. Ben-Shachar, Brenton M. Wiernik, Philip Waggoner, and Dominique Makowski. 2021. “see: An R Package for Visualizing Statistical Models. Journal of Open Source Software. 6(64):3393. 10.21105/joss.03393.

Luke SG. Evaluating Significance in Linear Mixed-Effects Models in r. Behavior Research Methods. 2017;49(4):1494–502. PubMed

Makowski, Dominique, Mattan S. Ben-Shachar, Indrajeet Patil, and Daniel Lüdecke. 2020. “Estimation of Model-Based Predictions, Contrasts and Means. CRAN. https://github.com/easystats/modelbased.

Masur, Philipp K., and Michael Scharkow. 2020. “specr: Conducting and Visualizing Specification Curve Analyses (Version 1.0.0).” https://CRAN.R-project.org/package=specr.

Meschiari, Stefano. 2022. Latex2exp: Use LaTeX Expressions in Plots. https://www.stefanom.io/latex2exp/.

Miles, C. 2008. “Testing Market-Based Instruments for Conservation in Northern Victoria.” Book Section. In Biodiversity: Integrating Conservation and Production: Case Studies from Australian Farms, Forests and Fisheries, edited by T. Norton, T. Lefroy, K. Bailey, and G. Unwin, 133–46. Melbourne, Australia: CSIRO Publishing.

Millard, Steven P. 2013. EnvStats: An r Package for Environmental Statistics. New York: Springer. https://www.springer.com.

Molina, Isabel, and Yolanda Marhuenda. 2015. “sae: An R Package for Small Area Estimation. The R Journal. 7(1):81–98. https://journal.r-project.org/archive/2015/RJ-2015-007/RJ-2015-007.pdf.

Morrissey, Michael B., and Graeme D. Ruxton. 2018. “Multiple Regression Is Not Multiple Regressions: The Meaning of Multiple Regression and the Non-Problem of Collinearity. Philosophy, Theory, and Practice in Biology. 10(3). 10.3998/ptpbio.16039257.0010.003.

Müller, Kirill. 2020. here: A Simpler Way to Find Your Files. https://here.r-lib.org/.

Nakagawa Shinichi, Cuthill Innes C. Effect Size, Confidence Interval and Statistical Significance: A Practical Guide for Biologists. Biological Reviews. 2007;82(4):591–605. 10.1111/j.1469-185X.2007.00027.x. PubMed

Nakagawa, Shinichi, Malgorzata Lagisz, Michael D. Jennions, Julia Koricheva, Daniel W. A. Noble, Timothy H. Parker, Alfredo Sánchez-Tójar, Yefeng Yang, and Rose E. O’Dea. 2022. “Methods for Testing Publication Bias in Ecological and Evolutionary Meta-Analyses. Methods in Ecology and Evolution. 13(1):4–21. 10.1111/2041-210X.13724.

Nakagawa, Shinichi, Malgorzata Lagisz, Rose E. O’Dea, Patrice Pottier, Joanna Rutkowska, Alistair M. Senior, Yefeng Yang, and Daniel W. A. Noble. 2023. “orchaRd 2.0: An r Package for Visualizing Meta-Analyses with Orchard Plots. EcoEvoRxiv. 12:4–12. 10.32942/X2QC7K.

Nakagawa Shinichi, Yang Yefeng, Macartney Erin L, Spake Rebecca, Lagisz Malgorzata. Quantitative Evidence Synthesis: A Practical Guide on Meta-Analysis, Meta-Regression, and Publication Bias Tests for Environmental Sciences. Environmental Evidence. 2023;12(1):8. 10.1186/s13750-023-00301-6. PubMed PMC

Nakagawa S, Noble DW, Senior AM, Lagisz M. Meta-Evaluation of Meta-Analysis: Ten Appraisal Questions for Biologists. BMC Biology. 2017;15(1):18. 10.1186/s12915-017-0357-7. PubMed PMC

Nicolaus M, Michler SPM, Ubels R, van der Velde M, Komdeur J, Both C, Tinbergen JM. Sex-Specific Effects of Altered Competition on Nestling Growth and Survival: An Experimental Manipulation of Brood Size and Sex Ratio. Journal of Animal Ecology. 2009;78(2):414–26. 10.1111/j.1365-2656.2008.01505.x. PubMed

Noble Daniel W. A, Lagisz Malgorzata, O’Dea Rose E, Nakagawa Shinichi. Nonindependence and Sensitivity Analyses in Ecological and Evolutionary Meta-Analyses. Molecular Ecology. 2017;26(9):2410–25. 10.1111/mec.14031. PubMed

O’Hara Robert B, Johan Kotze D. Do Not Log-Transform Count Data. Methods in Ecology and Evolution. 2010;1(2):118–22. 10.1111/j.2041-210x.2010.00021.x.

Open Science Collaboration. 2015. “Estimating the Reproducibility of Psychological Science.” Science. 349(6251):aac4716. 10.1126/science.aac4716. PubMed

Page Matthew J, Moher David. Evaluations of the Uptake and Impact of the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) Statement and Extensions: A Scoping Review. Systematic Reviews. 2017;6(1):263. 10.1186/s13643-017-0663-8. PubMed PMC

Parker Timothy H, Forstmeier Wolfgang, Koricheva Julia, Fidler Fiona, Hadfield Jarrod D, Chee Yung En, Kelly Clint D, Gurevitch Jessica, Nakagawa Shinichi. Transparency in Ecology and Evolution: Real Problems, Real Solutions. Trends in Ecology & Evolution. 2016;31(9):711–9. 10.1016/j.tree.2016.07.002. PubMed

Parker Timothy H, Yang Yefeng. Exaggerated Effects in Ecology. Nature Ecology & Evolution. 2023. 10.1038/s41559-023-02156-z. PubMed

Pedersen, Thomas Lin. 2024. patchwork: The Composer of Plots. https://patchwork.data-imaginist.com.

Pei Yifan, Forstmeier Wolfgang, Wang Daiping, Martin Katrin, Rutkowska Joanna, Kempenaers Bart. Proximate Causes of Infertility and Embryo Mortality in Captive Zebra Finches. The American Naturalist. 2020;196(5):577–96. 10.1086/710956. PubMed

Qiu, Yixuan. 2024. showtext: Using Fonts More Easily in r Graphs. https://github.com/yixuan/showtext.

R Core Team. 2024. R: A Language and Environment for Statistical Computing. Vienna, Austria: R Foundation for Statistical Computing. https://www.R-project.org/.

Rosenberg, M. S. 2013. “Moment and Least-Squares Based Approaches to Metaanalytic Inference.” Book Section. In Handbook of Meta-Analysis in Ecology and Evolution, edited by J. Koricheva, J. Gurevitch, and K. Mengersen, 108–24. Princeton, USA: Princeton University Press.

Royle NJ, Hartley IR, Owens IPF, Parker GA. Sibling Competition and the Evolution of Growth Rates in Birds. Proceedings of the Royal Society B-Biological Sciences. 1999;266(1422):923–32. 10.1098/rspb.1999.0725.

Scheinin, Ilari, Maria Kalimeri, Vilma Jagerroos, Juuso Parkkinen, Emmi Tikkanen, Peter Würtz, and Antti Kangas. 2020. ggforestplot: Forestplots of Measures of Effects and Their Confidence Intervals. https://github.com/NightingaleHealth/ggforestplot.

Schloerke, Barret, Di Cook, Joseph Larmarange, Francois Briatte, Moritz Marbach, Edwin Thoen, Amos Elberg, and Jason Crowley. 2024. GGally: Extension to “ggplot2”. https://ggobi.github.io/ggally/.

Schweinsberg M, Feldman M, Staub N, van den Akker OR, van Aert RCM, Malm van Assen Y, Liu, et al. Same Data, Different Conclusions: Radical Dispersion in Empirical Results When Independent Analysts Operationalize and Test the Same Hypothesis. Organizational Behavior and Human Decision Processes. 2021;165:228–49. 10.1016/j.obhdp.2021.02.003.

Senior Alistair M, Grueber Catherine E, Kamiya Tsukushi, Lagisz Malgorzata, O’Dwyer Katie, Santos Eduardo S. A, Nakagawa Shinichi. Heterogeneity in Ecological and Evolutionary Meta-Analyses: Its Magnitude and Implications. Ecology. 2016;97(12):3293–9. 10.1002/ecy.1591. PubMed

Shavit, A., and Aaron M. Ellison. 2017. Stepping in the Same River Twice: Replication in Biological Research. Edited Book. New Haven, Connecticut, USA: Yale University Press.

Siegel, Kyle R., Muskanjot Kaur, A. Calvin Grigal, Rebecca A. Metzler, and Gary H. Dickinson. 2022. “Meta-Analysis Suggests Negative, but pCO2-Specific, Effects of Ocean Acidification on the Structural and Functional Properties of Crustacean Biomaterials.” Ecology and Evolution. 12(6):e8922. 10.1002/ece3.8922. PubMed PMC

Silberzahn R, Uhlmann EL, Martin DP, Anselmi P, Aust F, Awtrey E, Bahník Š, et al. Many Analysts, One Data Set: Making Transparent How Variations in Analytic Choices Affect Results. Advances in Methods and Practices in Psychological Science. 2018;1(3):337–56. 10.1177/2515245917747646.

Silge, Julia, and David Robinson. 2016. “tidytext: Text Mining and Analysis Using Tidy Data Principles in r.” JOSS. 1(3). 10.21105/joss.00037.

Simons Daniel J, Shoda Yuichi, Stephen Lindsay D. Constraints on Generality (COG): A Proposed Addition to All Empirical Papers. Perspectives on Psychological Science. 2017. 10.1177/174569161770863. PubMed

Simonsohn, Uri, Joseph P. Simmons, and Leif D. Nelson. 2015. “Specification Curve: Descriptive and Inferential Statistics on All Reasonable Specifications.” Manuscript. SSRN Electronic Journal. 10.2139/ssrn.2694998.

Simonsohn Uri, Simmons Joseph P, Nelson Leif D. Specification Curve Analysis. Nature Human Behaviour. 2020;4(11):1208–14. 10.1038/s41562-020-0912-z. PubMed

Sjoberg, Daniel D., Karissa Whiting, Michael Curry, Jessica A. Lavery, and Joseph Larmarange. 2021. “Reproducible Summary Tables with the Gtsummary Package.” The R Journal. 13:570–80. 10.32614/RJ-2021-053.

Slowikowski, Kamil. 2024. ggrepel: Automatically Position Non-Overlapping Text Labels with “ggplot2”. https://ggrepel.slowkow.com/.

Stanton-Geddes, John, Cintia Gomes de Freitas, and Cristian de Sales Dambros. 2014. “In Defense of p Values: Comment on the Statistical Methods Actually Used by Ecologists.” Ecology. 95(3):637–42. 10.1890/13-1156.1. PubMed

Steegen Sara, Tuerlinckx Francis, Gelman Andrew, Vanpaemel Wolf. Increasing Transparency Through a Multiverse Analysis. Perspectives on Psychological Science. 2016;11(5):702–12. 10.1177/1745691616658637. PubMed

Taylor, James W., and Kathryn S. Taylor. 2023. “Combining Probabilistic Forecasts of COVID-19 Mortality in the United States.” European Journal of Operational Research. 304(1):25–41. 10.1016/j.ejor.2021.06.044. PubMed PMC

Tierney, Nicholas, and Dianne Cook. 2023. “Expanding Tidy Data Principles to Facilitate Missing Data Exploration, Visualization and Assessment of Imputations.” Journal of Statistical Software. 105(7):1–31. 10.18637/jss.v105.i07.

Touchon, Justin C., and Michael W. McCoy. 2016. “The Mismatch Between Current Statistical Practice and Doctoral Training in Ecology.” Ecosphere. 7(8):e01394. 10.1002/ecs2.1394.

Ushey, Kevin, and Hadley Wickham. 2023. renv: Project Environments. https://rstudio.github.io/renv/.

van den Brand, Teun. 2024. Ggh4x: Hacks for “ggplot2”. https://github.com/teunbrand/ggh4x.

Werf Vander, Eric. Lack’s Clutch Size Hypothesis: An Examination of the Evidence Using Meta-Analysis. Ecology. 1992;73(5):1699–705. 10.2307/1940021.

Hoef Ver, Jay M. Who Invented the Delta Method? The American Statistician. 2012;66(2):124–7. 10.1080/00031305.2012.687494.

Verhulst S, Holveck MJ, Riebel K. Long-Term Effects of Manipulated Natal Brood Size on Metabolic Rate in Zebra Finches. Biology Letters. 2006;2(3):478–80. 10.1098/rsbl.2006.0496. PubMed PMC

Vesk PA, Morris WK, McCallum W, Apted R, Miles C. Processes of Woodland Eucalypt Regeneration: Lessons from the Bush Returns Trial. Proceedings of the Royal Society of Victoria. 2016;128:54–63.

Viechtbauer, Wolfgang. 2010. “Conducting Meta-Analyses in R with the metafor Package.” Journal of Statistical Software. 36(3):1–48. 10.18637/jss.v036.i03.

Wickham, Hadley, Mara Averick, Jennifer Bryan, Winston Chang, Lucy D’Agostino McGowan, Romain François, Garrett Grolemund, et al. 2019. “Welcome to the tidyverse.” Journal of Open Source Software. 4(43):1686. 10.21105/joss.01686.

Wickham, Hadley, Jim Hester, Winston Chang, and Jennifer Bryan. 2022. devtools: Tools to Make Developing r Packages Easier. https://devtools.r-lib.org/.

Wickham, Hadley, Thomas Lin Pedersen, and Dana Seidel. 2023. scales: Scale Functions for Visualization. https://scales.r-lib.org.

Wilke, Claus O. 2024. cowplot: Streamlined Plot Theme and Plot Annotations for “ggplot2”. https://wilkelab.org/cowplot/.

Xie, Yihui. 2024a. knitr: A General-Purpose Package for Dynamic Report Generation in r. https://yihui.org/knitr/.

———. 2024b. xfun: Supporting Functions for Packages Maintained by “Yihui Xie”. https://github.com/yihui/xfun.

Yang Yefeng, Sánchez-Tójar Alfredo, O’Dea Rose E, Noble Daniel W. A, Koricheva Julia, Jennions Michael D, Parker Timothy H, Lagisz Malgorzata, Nakagawa Shinichi. Publication Bias Impacts on Effect Size, Statistical Power, and Magnitude (Type m) and Sign (Type s) Errors in Ecology and Evolutionary Biology. BMC Biology. 2023;21(1):71. 10.1186/s12915-022-01485-y. PubMed PMC

Zeileis, Achim, Jason C. Fisher, Kurt Hornik, Ross Ihaka, Claire D. McWhite, Paul Murrell, Reto Stauffer, and Claus O. Wilke. 2020. “colorspace: A Toolbox for Manipulating and Assessing Colors and Palettes.” Journal of Statistical Software. 96(1):1–49. 10.18637/jss.v096.i01.

Same data, different analysts: variation in effect sizes due to analytical decisions in ecology and evolutionary biology