• Something wrong with this record ?

Deciding what to replicate: A decision model for replication study selection under resource and knowledge constraints

PM. Isager, RCM. van Aert, Š. Bahník, MJ. Brandt, KA. DeSoto, R. Giner-Sorolla, JI. Krueger, M. Perugini, I. Ropovik, AE. van 't Veer, M. Vranka, D. Lakens

. 2023 ; 28 (2) : 438-451. [pub] 20211220

Language English Country United States

Document type Journal Article

Persistent link   https://www.medvik.cz/link/bmc23010611

Robust scientific knowledge is contingent upon replication of original findings. However, replicating researchers are constrained by resources, and will almost always have to choose one replication effort to focus on from a set of potential candidates. To select a candidate efficiently in these cases, we need methods for deciding which out of all candidates considered would be the most useful to replicate, given some overall goal researchers wish to achieve. In this article we assume that the overall goal researchers wish to achieve is to maximize the utility gained by conducting the replication study. We then propose a general rule for study selection in replication research based on the replication value of the set of claims considered for replication. The replication value of a claim is defined as the maximum expected utility we could gain by conducting a replication of the claim, and is a function of (a) the value of being certain about the claim, and (b) uncertainty about the claim based on current evidence. We formalize this definition in terms of a causal decision model, utilizing concepts from decision theory and causal graph modeling. We discuss the validity of using replication value as a measure of expected utility gain, and we suggest approaches for deriving quantitative estimates of replication value. Our goal in this article is not to define concrete guidelines for study selection, but to provide the necessary theoretical foundations on which such concrete guidelines could be built. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

References provided by Crossref.org

000      
00000naa a2200000 a 4500
001      
bmc23010611
003      
CZ-PrNML
005      
20230801132533.0
007      
ta
008      
230718s2023 xxu f 000 0|eng||
009      
AR
024    7_
$a 10.1037/met0000438 $2 doi
035    __
$a (PubMed)34928679
040    __
$a ABA008 $b cze $d ABA008 $e AACR2
041    0_
$a eng
044    __
$a xxu
100    1_
$a Isager, Peder Mortvedt $u Department of Industrial Engineering and Innovation Sciences, Eindhoven University of Technology $1 https://orcid.org/0000000269223590
245    10
$a Deciding what to replicate: A decision model for replication study selection under resource and knowledge constraints / $c PM. Isager, RCM. van Aert, Š. Bahník, MJ. Brandt, KA. DeSoto, R. Giner-Sorolla, JI. Krueger, M. Perugini, I. Ropovik, AE. van 't Veer, M. Vranka, D. Lakens
520    9_
$a Robust scientific knowledge is contingent upon replication of original findings. However, replicating researchers are constrained by resources, and will almost always have to choose one replication effort to focus on from a set of potential candidates. To select a candidate efficiently in these cases, we need methods for deciding which out of all candidates considered would be the most useful to replicate, given some overall goal researchers wish to achieve. In this article we assume that the overall goal researchers wish to achieve is to maximize the utility gained by conducting the replication study. We then propose a general rule for study selection in replication research based on the replication value of the set of claims considered for replication. The replication value of a claim is defined as the maximum expected utility we could gain by conducting a replication of the claim, and is a function of (a) the value of being certain about the claim, and (b) uncertainty about the claim based on current evidence. We formalize this definition in terms of a causal decision model, utilizing concepts from decision theory and causal graph modeling. We discuss the validity of using replication value as a measure of expected utility gain, and we suggest approaches for deriving quantitative estimates of replication value. Our goal in this article is not to define concrete guidelines for study selection, but to provide the necessary theoretical foundations on which such concrete guidelines could be built. (PsycInfo Database Record (c) 2023 APA, all rights reserved).
650    _2
$a lidé $7 D006801
650    _2
$a nejistota $7 D035501
650    12
$a znalosti $7 D019359
650    12
$a teoretické modely $7 D008962
655    _2
$a časopisecké články $7 D016428
700    1_
$a van Aert, Robbie C M $u Department of Methodology and Statistics, Tilburg University $1 https://orcid.org/0000000161870665
700    1_
$a Bahník, Štěpán $u Department of Management, Faculty of Business Administration, Prague University of Economics and Business $1 https://orcid.org/0000000205796808 $7 xx0194885
700    1_
$a Brandt, Mark J $u Department of Social Psychology, Tilburg University $1 https://orcid.org/0000000271857031
700    1_
$a DeSoto, K Andrew $u Association for Psychological Science $1 https://orcid.org/0000000190610301
700    1_
$a Giner-Sorolla, Roger $u School of Psychology, University of Kent $1 https://orcid.org/0000000266908842
700    1_
$a Krueger, Joachim I $u Department of Cognitive, Linguistic and Psychological Sciences, Brown University $1 https://orcid.org/0000000196071695
700    1_
$a Perugini, Marco $u Department of Psychology, University of Milano-Bicocca $1 https://orcid.org/0000000248646623
700    1_
$a Ropovik, Ivan $u Faculty of Education, Charles University $1 https://orcid.org/0000000152221233
700    1_
$a van 't Veer, Anna E $u Methodology and Statistics Unit, Institute of Psychology, Leiden University $1 https://orcid.org/0000000227331841
700    1_
$a Vranka, Marek $u Faculty of Social Sciences, Charles University $1 https://orcid.org/0000000334139062 $7 hka2017945226
700    1_
$a Lakens, Daniël $u Department of Industrial Engineering and Innovation Sciences, Eindhoven University of Technology $1 https://orcid.org/000000020247239X
773    0_
$w MED00005597 $t Psychological methods $x 1939-1463 $g Roč. 28, č. 2 (2023), s. 438-451
856    41
$u https://pubmed.ncbi.nlm.nih.gov/34928679 $y Pubmed
910    __
$a ABA008 $b sig $c sign $y p $z 0
990    __
$a 20230718 $b ABA008
991    __
$a 20230801132530 $b ABA008
999    __
$a ok $b bmc $g 1963197 $s 1196876
BAS    __
$a 3
BAS    __
$a PreBMC-MEDLINE
BMC    __
$a 2023 $b 28 $c 2 $d 438-451 $e 20211220 $i 1939-1463 $m Psychological methods $n Psychol Methods $x MED00005597
LZP    __
$a Pubmed-20230718

Find record

Citation metrics

Archiving options