• Je něco špatně v tomto záznamu ?

Automation bias in medicine: The influence of automated diagnoses on interpreter accuracy and uncertainty when reading electrocardiograms

RR. Bond, T. Novotny, I. Andrsova, L. Koc, M. Sisakova, D. Finlay, D. Guldenring, J. McLaughlin, A. Peace, V. McGilligan, SJ. Leslie, H. Wang, M. Malik,

. 2018 ; 51 (6S) : S6-S11. [pub] 20180810

Jazyk angličtina Země Spojené státy americké

Typ dokumentu časopisecké články

Perzistentní odkaz   https://www.medvik.cz/link/bmc19045294

INTRODUCTION: Interpretation of the 12‐lead Electrocardiogram (ECG) is normally assisted with an automated diagnosis (AD), which can facilitate an 'automation bias' where interpreters can be anchored. In this paper, we studied, 1) the effect of an incorrect AD on interpretation accuracy and interpreter confidence (a proxy for uncertainty), and 2) whether confidence and other interpreter features can predict interpretation accuracy using machine learning. METHODS: This study analysed 9000 ECG interpretations from cardiology and non-cardiology fellows (CFs and non-CFs). One third of the ECGs involved no ADs, one third with ADs (half as incorrect) and one third had multiple ADs. Interpretations were scored and interpreter confidence was recorded for each interpretation and subsequently standardised using sigma scaling. Spearman coefficients were used for correlation analysis and C5.0 decision trees were used for predicting interpretation accuracy using basic interpreter features such as confidence, age, experience and designation. RESULTS: Interpretation accuracies achieved by CFs and non-CFs dropped by 43.20% and 58.95% respectively when an incorrect AD was presented (p < 0.001). Overall correlation between scaled confidence and interpretation accuracy was higher amongst CFs. However, correlation between confidence and interpretation accuracy decreased for both groups when an incorrect AD was presented. We found that an incorrect AD disturbs the reliability of interpreter confidence in predicting accuracy. An incorrect AD has a greater effect on the confidence of non-CFs (although this is not statistically significant it is close to the threshold, p = 0.065). The best C5.0 decision tree achieved an accuracy rate of 64.67% (p < 0.001), however this is only 6.56% greater than the no-information-rate. CONCLUSION: Incorrect ADs reduce the interpreter's diagnostic accuracy indicating an automation bias. Non-CFs tend to agree more with the ADs in comparison to CFs, hence less expert physicians are more effected by automation bias. Incorrect ADs reduce the interpreter's confidence and also reduces the predictive power of confidence for predicting accuracy (even more so for non-CFs). Whilst a statistically significant model was developed, it is difficult to predict interpretation accuracy using machine learning on basic features such as interpreter confidence, age, reader experience and designation.

Citace poskytuje Crossref.org

000      
00000naa a2200000 a 4500
001      
bmc19045294
003      
CZ-PrNML
005      
20250610100118.0
007      
ta
008      
200109s2018 xxu f 000 0|eng||
009      
AR
024    7_
$a 10.1016/j.jelectrocard.2018.08.007 $2 doi
035    __
$a (PubMed)30122457
040    __
$a ABA008 $b cze $d ABA008 $e AACR2
041    0_
$a eng
044    __
$a xxu
100    1_
$a Bond, Raymond R $u Faculty of Computing and Engineering, Ulster University, Newtownabbey, United Kingdom. Electronic address: rb.bond@ulster.ac.uk.
245    10
$a Automation bias in medicine: The influence of automated diagnoses on interpreter accuracy and uncertainty when reading electrocardiograms / $c RR. Bond, T. Novotny, I. Andrsova, L. Koc, M. Sisakova, D. Finlay, D. Guldenring, J. McLaughlin, A. Peace, V. McGilligan, SJ. Leslie, H. Wang, M. Malik,
520    9_
$a INTRODUCTION: Interpretation of the 12‐lead Electrocardiogram (ECG) is normally assisted with an automated diagnosis (AD), which can facilitate an 'automation bias' where interpreters can be anchored. In this paper, we studied, 1) the effect of an incorrect AD on interpretation accuracy and interpreter confidence (a proxy for uncertainty), and 2) whether confidence and other interpreter features can predict interpretation accuracy using machine learning. METHODS: This study analysed 9000 ECG interpretations from cardiology and non-cardiology fellows (CFs and non-CFs). One third of the ECGs involved no ADs, one third with ADs (half as incorrect) and one third had multiple ADs. Interpretations were scored and interpreter confidence was recorded for each interpretation and subsequently standardised using sigma scaling. Spearman coefficients were used for correlation analysis and C5.0 decision trees were used for predicting interpretation accuracy using basic interpreter features such as confidence, age, experience and designation. RESULTS: Interpretation accuracies achieved by CFs and non-CFs dropped by 43.20% and 58.95% respectively when an incorrect AD was presented (p < 0.001). Overall correlation between scaled confidence and interpretation accuracy was higher amongst CFs. However, correlation between confidence and interpretation accuracy decreased for both groups when an incorrect AD was presented. We found that an incorrect AD disturbs the reliability of interpreter confidence in predicting accuracy. An incorrect AD has a greater effect on the confidence of non-CFs (although this is not statistically significant it is close to the threshold, p = 0.065). The best C5.0 decision tree achieved an accuracy rate of 64.67% (p < 0.001), however this is only 6.56% greater than the no-information-rate. CONCLUSION: Incorrect ADs reduce the interpreter's diagnostic accuracy indicating an automation bias. Non-CFs tend to agree more with the ADs in comparison to CFs, hence less expert physicians are more effected by automation bias. Incorrect ADs reduce the interpreter's confidence and also reduces the predictive power of confidence for predicting accuracy (even more so for non-CFs). Whilst a statistically significant model was developed, it is difficult to predict interpretation accuracy using machine learning on basic features such as interpreter confidence, age, reader experience and designation.
650    _2
$a srdeční arytmie $x diagnóza $7 D001145
650    12
$a automatizace $7 D001331
650    _2
$a zkreslení výsledků (epidemiologie) $7 D015982
650    12
$a klinické kompetence $7 D002983
650    _2
$a rozhodovací stromy $7 D003663
650    _2
$a chybná diagnóza $x statistika a číselné údaje $7 D003951
650    12
$a elektrokardiografie $7 D004562
650    _2
$a lidé $7 D006801
650    _2
$a odchylka pozorovatele $7 D015588
650    _2
$a nejistota $7 D035501
655    _2
$a časopisecké články $7 D016428
700    1_
$a Novotný, Tomáš, $d 1969- $u Department of Internal Medicine and Cardiology, University Hospital Brno and Faculty of Medicine of Masaryk University, Brno, Czechia. $7 xx0055126
700    1_
$a Andrsova, Irena $u Department of Internal Medicine and Cardiology, University Hospital Brno and Faculty of Medicine of Masaryk University, Brno, Czechia.
700    1_
$a Koc, Lumir $u Department of Internal Medicine and Cardiology, University Hospital Brno and Faculty of Medicine of Masaryk University, Brno, Czechia.
700    1_
$a Sisakova, Martina $u Department of Internal Medicine and Cardiology, University Hospital Brno and Faculty of Medicine of Masaryk University, Brno, Czechia.
700    1_
$a Finlay, Dewar $u Faculty of Computing and Engineering, Ulster University, Newtownabbey, United Kingdom.
700    1_
$a Guldenring, Daniel $u Faculty of Computing and Engineering, Ulster University, Newtownabbey, United Kingdom.
700    1_
$a McLaughlin, James $u Faculty of Computing and Engineering, Ulster University, Newtownabbey, United Kingdom.
700    1_
$a Peace, Aaron $u Department of Cardiology, Altnagelvin Hospital, Western Health and Social Care Trust, United Kingdom.
700    1_
$a McGilligan, Victoria $u Northern Ireland Centre for Stratified Medicine, C-TRIC, Ulster University, Derry~Londonderry, United Kingdom.
700    1_
$a Leslie, Stephen J $u Cardiac Unit, Raigmore Hospital, Inverness, UK, IV2 3UJ.
700    1_
$a Wang, Hui $u Faculty of Computing and Engineering, Ulster University, Newtownabbey, United Kingdom.
700    1_
$a Malik, Marek $u St. Paul's Cardiac Electrophysiology and Imperial College London, London, United Kingdom.
773    0_
$w MED00010003 $t Journal of electrocardiology $x 1532-8430 $g Roč. 51, č. 6S (2018), s. S6-S11
856    41
$u https://pubmed.ncbi.nlm.nih.gov/30122457 $y Pubmed
910    __
$a ABA008 $b sig $c sign $y a $z 0
990    __
$a 20200109 $b ABA008
991    __
$a 20250610100111 $b ABA008
999    __
$a ok $b bmc $g 1483563 $s 1083967
BAS    __
$a 3
BAS    __
$a PreBMC
BMC    __
$a 2018 $b 51 $c 6S $d S6-S11 $e 20180810 $i 1532-8430 $m Journal of electrocardiology $n J Electrocardiol $x MED00010003
LZP    __
$a Pubmed-20200109

Najít záznam

Citační ukazatele

Pouze přihlášení uživatelé

Možnosti archivace