Inter-observer variability
Dotaz
Zobrazit nápovědu
Aim. To determine the inter-observer reproducibility of 15 tests used for predicting difficult tracheal intubation (DI). Material and methods. Following local ethics committee approval and informed consent, 101 volunteers were examined by two assessors using 15 tests for predicting DI. The two assessors who were blinded to the results of the other, examined each volunteer independently. Cohen's kappa (ę) or first-order agreement coefficient (AC1) were used to measure agreement between assessor ratings on a qualitative scale. Agreement between two quantitative outcomes was described using the intraclass correlation coefficient (ICC) and Pearson's (PCC) or Spearman's (SCC) correlation coefficients. The following interpretation of the coefficients was used: poor (< 0.20), fair (0.21–0.40), satisfactory (0.41–0.60), good (0.61–0.80), and excellent (0.81–1.00). Results. Respective coefficients of inter-rater agreement and correlation coefficients were determined for the following parameters: pathologies associated with DI (ę=0.662, AC1=0.990), clinical impression (ę=-0.013, AC1=0.969), modified Mallampati test (ę=0.503, AC1=0.861), upper lip bite test (ę=0.370, AC1=0.897), temporo-mandibular joint movement (ę=0.088, AC1=0.797), max. anteroflexion of C-spine (ICC=0.136, SCC=0.391), max. retroflexion of C-spine (ICC=0.020, SCC=0.284), mandibular length (ICC=0.301, SCC=0.553), neck circumference (ICC=0.832, SCC=0.928), hyo-mental distance (ICC=0.378, SCC=0.472), thyro-mental distance (ICC=–0.002, PCC=0.265), sternomental distance (ICC=0.674, PCC=0.815), and finally, inter-incisor gap (ICC=0.695, PCC=0.785). Two tests (positive history of DI and retrogenia), were excluded from calculation because no positive cases were found. Conclusion. Best inter-rater agreement was found for the assessment of neck circumference while the highest discrepancies between raters were in goniometrically-measured mobility of the C-spine. Many of the pre-operative airway tests had only fair inter-observer reproducibility. This may be one reason why models for predicting difficult intubation are not universally reliable.
- MeSH
- antropometrie MeSH
- experimenty na lidech MeSH
- financování organizované MeSH
- intratracheální intubace metody normy škodlivé účinky MeSH
- lidé MeSH
- odchylka pozorovatele MeSH
- pravděpodobnostní funkce MeSH
- předoperační péče metody MeSH
- reprodukovatelnost výsledků MeSH
- rizikové faktory MeSH
- sexuální faktory MeSH
- Check Tag
- lidé MeSH
Práce stručně shrnuje výsledky studií pracovní skupiny EFLM o biologických variabilitách - EuBIVAS. Jsou popsány metody výpočtu intra a inter individuálních variabilit. Předmětem sdělení jsou dále aplikace biologických variabilit pro stanovení kritérií analytické kvality (kalkulace hodnot APS a nejistot měření) a pro stanovení hodnot významnosti změn dvou následných měření (RCV) jako kritéria schopnosti monitorovat průběh terapie u pacientů. Je popsána databáze EFLM biologických variabilit a trendy použití výsledků měření klinických laboratoří k personalizaci a individualizaci jejich interpretací (individuální referenční intervaly, přístupy machine learning).
The publication deals with a brief summarizing the results of a working group of EFLM for biological variation studies (Eu- BIVAS). There are described namely calculations of within- and between subject biological variation and their applications for calculation of analytical quality criteria (APS values) and for calculation of significance of reference changes values as potency for monitoring of the patient´s state. We described EFLM biological variation database. We also show the necessity of biological variation for personalized medicine and individualized approach to laboratory results (individual reference intervals, machine learning).
Cílem práce bylo porovnání úspěšnosti texturního klasifikátoru a vyšetřujícího lékaře (radiologa) při diagnóze autoimunitní thyroiditidy ze sonografického obrazu snímaného v B-módu. Určení inter- a intrapersonální variability lékařů. Datový soubor obsahující 161 vyšetřovaných subjektů byl rozdělen do tří skupin dle celkového klinického vyšetření: normální – H (healthy); hraniční stav – BS (border state); autoimunitní thyroiditida – AT. Následně byl soubor čtyřmi vyšetřujícími lékaři a Bayesovským klasifikátorem, založeným na texturních příznacích, hodnocen do těchto skupin. Dva lékaři dosáhli vyšší úspěšnosti při hodnocení subjektů z normální skupiny (74,4 % a 83,3 %) a dva lékaři hodnotili lépe subjekty s autoimunitní thyroiditidou (59,0% a 77,4 %). Klasifikátor dosáhl relativně vysoké a vyrovnané úspěšnosti pro obě tyto skupiny (100,0 % pro normální a 87,5 % pro thyroiditidu). Rozdílný úspěch jednotlivých lékařů při hodnocení subjektů vyústil ve vyšší interpersonální variabilitu, tedy nízkou shodu mezi nimi. V intrapersonální variabilitě jednotlivých lékařů nebyl nalezen významný rozdíl. Vzhledem ke slabé shodě mezi vyšetřujícími lékaři při diagnostice autoimunitní thyroiditidy ze sonografických obrazů a vysoké a vyrovnané úspěšnosti klasifikátoru se zdá jako nejvýhodnější pro stanovení konečné diagnózy kombinace automatické klasifikace obrazů a klinických zkušeností lékařů.
The objective has been to compare success of the texture classifier and a human observer in diagnosis of the autoimmune thyroiditis from B-mode ultrasound images and to determine inter- and intra-observer variability. The data set of 161 subjects was classified by four human observers and by the Bayes classifier based on the texture features to three classes (healthy, border state, autoimmune thyroiditis). Two observers had a higher success rate when classifying the healthy class (74.4 % and 83.3 %), the other two observers classified better cases with autoimmune thyroiditis (59.0 % and 77.4 %). The classifier gave the relatively high and balanced success rate for both classes (100.0 % for healthy and 875 % for thyroiditis). The different observers’ success rates resulted in the high inter-observer variability, showing only a fair agreement among the human observers. There was no significant difference among human observers in the intra-observer variability. Due to the fair agreement among observers in the diagnosis of autoimmune thyroiditis from ultrasound images and good results of the classifier, the best way in establishing diagnosis is computer-aided diagnosis combined with observers’ clinical experience.
- Klíčová slova
- sonografický obraz, B-mode sonografie, texturní analýza, počítačem podporovaná diagnóza, interpersonální variabilita, koeficient Kappa, vážený koeficient Kappa,
- MeSH
- autoimunitní tyreoiditida diagnóza ultrasonografie MeSH
- financování organizované MeSH
- interpretace obrazu počítačem přístrojové vybavení využití MeSH
- interpretace statistických dat MeSH
- lidé MeSH
- odchylka pozorovatele MeSH
- reprodukovatelnost výsledků MeSH
- senzitivita a specificita MeSH
- štítná žláza MeSH
- ultrasonografie metody statistika a číselné údaje MeSH
- Check Tag
- lidé MeSH
Excessive time for analysis may impede microcirculatory studies with large amounts of video data. Engaging more personnel in the analyses seems to be a rational approach in that scenario and could shorten the time-interval between capturing images and obtaining results. Our hypothesis was that novice users would be able to determine standard microcirculatory parameters using a semi-automated software with an acceptable degree of variability after participating in a standardized interactive training session. 14 volunteers were included in the study. All volunteers analyzed separately the same sample video after the training. The kappa statistic was calculated for the primary outcome parameter microvascular flow index (MFI) within small and large vessels and indicated a fair level of agreement in the results of the novice users. A standardized interactive tutorial can be useful to teach microcirculatory analysis in previously untrained subjects.
- MeSH
- audiovizuální záznam MeSH
- automatizace MeSH
- kardiologie výchova MeSH
- lidé MeSH
- mikrocirkulace * MeSH
- odchylka pozorovatele MeSH
- software MeSH
- statistické modely MeSH
- ústa krevní zásobení MeSH
- Check Tag
- lidé MeSH
- Publikační typ
- časopisecké články MeSH
- multicentrická studie MeSH
- práce podpořená grantem MeSH
- Geografické názvy
- Evropa MeSH
- Kanada MeSH
This study was conducted to determine the incidence of grey-white matter abnormalities (GWMAs) on magnetic resonance images (MRIs) in patients with hippocampal sclerosis (HS), to assess the inter-observer reliability of this finding, and to establish a possible relationship between GWMA and histopathological findings in the anterior part of the temporal lobe, as well as its other relation to clinical variables. We established a group of 55 patients with histologically proven HS. Three observers independently reviewed the MRIs to assess whether GWMA was present. Substantial independent inter-observer agreement was reached for 44 of the 55 patients (80%) (Fleiss' kappa 0.732; p<0.0001). GWMAs were present in 38% of patients (HS+GWMA). Focal cortical dysplasia (FCD) of type IIIa (ILAE classification) was present in 31% of patients. FCD type IIIa was present in 52.4% with HS+GWMA, and in 17.6% without GWMA (HS-GWMA) (p=0.007). We did not find any statistically significant differences in the postoperative outcomes between HS+GWMA and HS-GWMA. We did not find any statistically significant differences in the presence or absence of GWMA and FCD of the temporal pole in relation to the onset of epilepsy, the duration of epilepsy, or the presence of potential epileptogenic insults. GWMA in the anterior part of temporal lobe in patients with HS is a reliable assessment sign for observers who are experienced in evaluating the MRIs of epilepsy patients. The presence of GWMA is significantly associated with the presence of FCD type IIIa in these patients. The presence or absence of GWMA and FCD type IIIa does not influence the postoperative outcome of HS patients.
- MeSH
- anatomická značka patologie MeSH
- dítě MeSH
- dospělí MeSH
- epilepsie temporálního laloku patologie chirurgie MeSH
- hipokampus patologie MeSH
- laboratorní medicína metody normy statistika a číselné údaje MeSH
- lidé středního věku MeSH
- lidé MeSH
- magnetická rezonanční tomografie metody normy statistika a číselné údaje MeSH
- malformace mozkové kůry patologie chirurgie MeSH
- mladiství MeSH
- mladý dospělý MeSH
- odchylka pozorovatele MeSH
- retrospektivní studie MeSH
- skleróza patologie MeSH
- spánkový lalok patologie MeSH
- Check Tag
- dítě MeSH
- dospělí MeSH
- lidé středního věku MeSH
- lidé MeSH
- mladiství MeSH
- mladý dospělý MeSH
- mužské pohlaví MeSH
- ženské pohlaví MeSH
- Publikační typ
- časopisecké články MeSH
- práce podpořená grantem MeSH
- validační studie MeSH
The European LeukemiaNet MDS (EUMDS) registry is collecting data of myelodysplastic syndrome (MDS) patients belonging to the IPSS low or intermediate-1 category, newly diagnosed by local cytologists. The diagnosis of MDS can be challenging, and some data report inter-observer variability with regard to the assessment of the MDS subtype. In order to ensure that correct diagnoses were made by the participating centres, blood and bone marrow slides of 10% of the first 1000 patients were reviewed by an 11-person panel of cytomorphologists. All slides were rated by at least 3 panel members (median 8 panel members; range 3-9). Marrow slides from 98 out of 105 patients were of good quality and therefore could be rated properly according to the WHO 2001 classification, including assessment of dysplastic lineages. The agreement between the reviewers whether the diagnosis was MDS or non-MDS was strong with an intra-class correlation coefficient (ICC) of 0.85. Six cases were detected not to fit the entry criteria of the registry, because they were diagnosed uniformly as CMML or AML by the panel members. The agreement by WHO 2001 classification was strong as well (ICC = 0.83). The concordance of the assessment of dysplastic lineages was substantial for megakaryopoiesis and myelopoiesis and moderate for erythropoiesis. Our data show that in general, the inter-observer agreement was high and a very low percentage of misdiagnosed cases had been entered into the EUMDS registry. Further studies including histomorphology are warranted.
- MeSH
- cytodiagnostika metody normy MeSH
- dospělí MeSH
- lidé středního věku MeSH
- lidé MeSH
- mladý dospělý MeSH
- myelodysplastické syndromy krev diagnóza MeSH
- odchylka pozorovatele * MeSH
- registrace statistika a číselné údaje MeSH
- reprodukovatelnost výsledků MeSH
- senioři nad 80 let MeSH
- senioři MeSH
- senzitivita a specificita MeSH
- vyšetřování kostní dřeně metody normy MeSH
- Check Tag
- dospělí MeSH
- lidé středního věku MeSH
- lidé MeSH
- mladý dospělý MeSH
- mužské pohlaví MeSH
- senioři nad 80 let MeSH
- senioři MeSH
- ženské pohlaví MeSH
- Publikační typ
- časopisecké články MeSH
OBJECTIVE: The diagnosis of arterial occlusion has a considerable impact on the indication of mechanical thrombectomy, and CT angiography (CTA) is recommended in the management of acute stroke. The goal of the present study is to assess the interrater agreement in the diagnosis of occlusion of intracranial arteries on CTA between a neuroradiologist and neurologists. METHODS: CTA images of 75 acute stroke patients were evaluated for occlusion of intracranial arteries by an experienced interventional neuroradiologist, and stroke and general neurologists. RESULTS: 75 patients who were treated by intravenous thrombolysis were enrolled in the study. CTA images were available for all 75 patients (34 females; mean age ± SD, 72 ± 14 years; National Institutes of Health Stroke Scale 10; median 8-14; and Alberta Stroke Program Early CT mean 9.7). The agreement between the neuroradiologist and neurologists in evaluation of intracranial artery occlusion was as follows: occlusion of the middle cerebral artery segment M1: observer agreement 77%, kappa (κ) = 0.61 and middle cerebral artery M2: observer agreement 77%, κ 0.48; internal carotid artery: observer agreement 92%, κ 0.84; T occlusion: observer agreement 90.0%, κ 0.33; posterior cerebral artery segments P1 and P2: observer agreement 98%, κ 0.97; basilar artery: observer agreement 96%, κ 0.92; and vertebral artery segment V4: observer agreement 88%, κ 0.48. CONCLUSION: Interrater agreement of CTA evaluation of occlusion between the neurologists and the neuroradiologist was very strong. The ability of the trained neurologists to read an intracranial large vessel occlusion correctly may improve the door-to-needle times in acute stroke. Advances in knowledge: In this study, the neurologists were able to recognize occlusion of intracranial arteries. This could accelerate the management of acute stroke care.
- MeSH
- cévní mozková příhoda diagnostické zobrazování MeSH
- CT angiografie metody statistika a číselné údaje MeSH
- lidé MeSH
- mozek diagnostické zobrazování MeSH
- mozková angiografie metody statistika a číselné údaje MeSH
- neurologové statistika a číselné údaje MeSH
- neuroradiografie statistika a číselné údaje MeSH
- odchylka pozorovatele MeSH
- reprodukovatelnost výsledků MeSH
- senioři MeSH
- Check Tag
- lidé MeSH
- mužské pohlaví MeSH
- senioři MeSH
- ženské pohlaví MeSH
- Publikační typ
- časopisecké články MeSH
The most common approach to assess fetal well-being during delivery is monitoring of fetal heart rate and uterine contractions-the cardiotocogram (CTG). Nevertheless, 40years since the introduction of CTG to clinical practice, its evaluation is still challenging with high inter- and intra-observer variability. Therefore the development of more objective methods has become an issue of major importance in the field. Unlike the usually proposed approaches to assign classes for classification methods that rely either on biochemical parameters (e.g. pH value) or a simple aggregation of expert judgment, this work investigates the use of an alternative labeling system using latent class analysis (LCA) along with an ordinal classification scheme. The study is performed on a well-documented open-access database, where nine expert obstetricians provided CTG annotations. The LCA is proposed here to produce more objective class labels while the ordinal classification aims to explore the natural ordering, and representation of increased severity, for obtaining the final results. The results are promising suggesting that more effort should be put into this proposed approach.
This study assessed the measurement error of a monitoring system, the Video Manual Motion Tracker 1.0 (VMMT1.0), during time-motion analysis of basketball players. In this study, four reliability parameters were used to assess the measurement error of the system: the systematic bias, the inter-observer reliability, the intra-observer reliability and the absolute reliability. A basketball game video was used for the analysis. To assess the inter-observer reliability, two observers analysed a player's covered distance for 50 different periods of the game. To assess the relative and absolute reliability of the covered distance, the chosen players were monitored three times by 41 qualified observers. The findings did not indicate a significant systematic bias in the measurement error using the VMMT1.0 (one-way ANOVA, P > 0.05). The intra-observer reliability of the monitoring system was rated as very high (intraclass correlation, ICC = 0.999), similar to its inter-observer reliability (Pearson product-moment correlation, r = 0.994). The absolute reliability does not appear to be significant (standard error of measurement, SEM = 0.34 m). The results showed that the measurement error of the VMMT1.0 is acceptable and comparable with that of other time-motion analysis techniques.