Interobserver and intraobserver agreement
Dotaz
Zobrazit nápovědu
- MeSH
- čtyřhlavý sval stehenní * diagnostické zobrazování MeSH
- kritický stav * MeSH
- lidé MeSH
- odchylka pozorovatele MeSH
- ultrasonografie MeSH
- Check Tag
- lidé MeSH
- Publikační typ
- časopisecké články MeSH
OBJECTIVE: To evaluate interobserver agreement for the assessment of local tumor extension in women with cervical cancer, among experienced and less experienced observers, using transvaginal ultrasound (TVS) and magnetic resonance imaging (MRI). METHODS: The TVS observers were all gynecologists and consultant ultrasound specialists, six with and seven without previous experience in cervical cancer imaging. The MRI observers were five radiologists experienced in pelvic MRI and four less experienced radiology residents without previous experience in MRI of the pelvis. The less experienced TVS observers and all MRI observers underwent a short basic training session in the assessment of cervical tumor extension, while the experienced TVS observers received only a written directive. All observers were assigned the same images from cervical cancer patients at all stages (n = 60) and performed offline evaluation to answer the following three questions: (1) Is there a visible primary tumor? (2) Does the tumor infiltrate > ⅓ of the cervical stroma? and (3) Is there parametrial invasion? Interobserver agreement within the four groups of observers was assessed using Fleiss kappa (κ) with 95% CI. RESULTS: Experienced and less experienced TVS observers, respectively, had moderate interobserver agreement with respect to tumor detection (κ (95% CI), 0.46 (0.40-0.53) and 0.46 (0.41-0.52)), stromal invasion > ⅓ (κ (95% CI), 0.45 (0.38-0.51) and 0.53 (0.40-0.58)) and parametrial invasion (κ (95% CI), 0.57 (0.51-0.64) and 0.44 (0.39-0.50)). Experienced MRI observers had good interobserver agreement with respect to tumor detection (κ (95% CI), 0.70 (0.62-0.78)), while less experienced MRI observers had moderate agreement (κ (95% CI), 0.51 (0.41-0.62)), and both experienced and less experienced MRI observers, respectively, had good interobserver agreement regarding stromal invasion (κ (95% CI), 0.80 (0.72-0.88) and 0.71 (0.61-0.81)) and parametrial invasion (κ (95% CI), 0.69 (0.61-0.77) and 0.71 (0.61-0.81)). CONCLUSIONS: We found interobserver agreement for the assessment of local tumor extension in patients with cervical cancer to be moderate for TVS and moderate-to-good for MRI. The level of interobserver agreement was associated with experience among TVS observers only for parametrial invasion. © 2021 The Authors. Ultrasound in Obstetrics & Gynecology published by John Wiley & Sons Ltd on behalf of International Society of Ultrasound in Obstetrics and Gynecology.
- Klíčová slova
- MRI, neoplasm staging, observer variation, ultrasonography, uterine cervical neoplasm,
- MeSH
- cervix uteri diagnostické zobrazování MeSH
- dospělí MeSH
- gynekologie statistika a číselné údaje MeSH
- klinické kompetence statistika a číselné údaje MeSH
- lidé středního věku MeSH
- lidé MeSH
- magnetická rezonanční tomografie metody statistika a číselné údaje MeSH
- nádory děložního čípku diagnostické zobrazování patologie MeSH
- odchylka pozorovatele MeSH
- radiologie statistika a číselné údaje MeSH
- reprodukovatelnost výsledků MeSH
- staging nádorů metody statistika a číselné údaje MeSH
- ultrasonografie metody statistika a číselné údaje MeSH
- vagina diagnostické zobrazování MeSH
- Check Tag
- dospělí MeSH
- lidé středního věku MeSH
- lidé MeSH
- ženské pohlaví MeSH
- Publikační typ
- časopisecké články MeSH
- hodnotící studie MeSH
Tubular adenoma (TA) and syringocystadenoma papilliferum (SCAP) may show histopathological overlap, with some lesions having features of both neoplasms (SCAP + TA). TA has been recently suggested to represent a carcinoma. Four observers blindly assessed 67 cases of TA, SCAP, and their lookalikes (poroma, apocrine adenoma, apocrine carcinoma; all lesions focally featuring a pseudopapillary pattern), and classified the lesions into one of four categories: (1) TA, (2) SCAP, (3) SCAP + TA, and (4) others. Lesions were also classified as benign or malignant. In only 29 cases was there unanimous agreement among the four observers, who classified 22 lesions as TA, three as SCAP, and four cases as others. Of the 38 cases where there was interobserver diagnostic variation, in 30, the diagnosis varied between TA or SCAP or SCAP + TA; the remainder fell in the others category. Analysis of the factors leading to interobserver variability indicated that diagnostic problems occurred when there were any of the following: epidermal acanthosis, papillomatosis, connection of the neoplastic tubules to the overlying epidermis and/or follicular infundibula, and plasma cell infiltration. These features accounted for the morphological overlap between TA and SCAP. All observers agreed that the lesions were benign; the only apocrine carcinoma included was recognized as such by all observers. From the study, it was concluded that TA may arise in the deep dermis without any epidermal connection, or, in other cases, it may be more superficially located with or without an epidermal connection. It may be reasonably inferred that, possibly as a response to infection, there may be accompanying plasma cells and variable acanthosis and papillomatosis, such that the appearances are those of "pure" SCAP, or lesions may have features "intermediate" or overlapping between TA and SCAP.
- MeSH
- adenom potní žlázy klasifikace diagnóza MeSH
- cystadenom klasifikace diagnóza MeSH
- dermatologie metody MeSH
- diferenciální diagnóza MeSH
- dospělí MeSH
- lidé středního věku MeSH
- lidé MeSH
- mladiství MeSH
- mnohočetné primární nádory diagnóza MeSH
- nádory potních žláz klasifikace diagnóza MeSH
- odchylka pozorovatele MeSH
- patologie metody MeSH
- senioři nad 80 let MeSH
- senioři MeSH
- Check Tag
- dospělí MeSH
- lidé středního věku MeSH
- lidé MeSH
- mladiství MeSH
- mužské pohlaví MeSH
- senioři nad 80 let MeSH
- senioři MeSH
- ženské pohlaví MeSH
- Publikační typ
- časopisecké články MeSH
- multicentrická studie MeSH
RATIONALE, AIMS AND OBJECTIVES: To evaluate obstetricians' inter- and intra-observer agreement on intrapartum cardiotocogram (CTG) recordings and to examine obstetricians' evaluations with respect to umbilical artery pH and base deficit. METHODS: Nine experienced obstetricians annotated 634 intrapartum CTG recordings. The evaluation of each recording was divided into four steps: evaluation of two 30-minute windows in the first stage of labour, evaluation of one window in the second stage of labour and labour outcome prediction. The complete set of evaluations used for this experiment is available online. The inter- and intra-observer agreement was evaluated using proportion of agreement and kappa coefficient. Clinicians' sensitivity and specificity was computed with respect to umbilical artery pH, base deficit and to Apgar score at the fifth minute. RESULTS: The overall proportion of agreement between clinicians reached 48% with 95% confidence intervals (CI) (CI: 47-50). Regarding the different classes, proportion of agreement ranged from 57% (CI: 54-60) for normal to 41% (CI: 36-46) for pathological class. The sensitivity of clinicians' majority vote to objective outcome was 39% (CI: 16-63) for the umbilical artery base deficit and 27% (CI: 16-42) for pH. The specificity was 89% (CI: 86-92) for both types of objective outcome. CONCLUSIONS: The reported inter-/intra-observer variability is large and this holds irrespective of clinicians' experience or work place. The results support the need of modernized guidelines for CTG evaluation and/or objectivization and repeatability by introduction of a computerized approach that could standardize the process of CTG evaluation within the delivery ward.
- Klíčová slova
- cardiotocography, fetal heart rate, inter-observer variability, intra-observer variability,
- MeSH
- kardiotokografie statistika a číselné údaje MeSH
- klinické kompetence * MeSH
- koncentrace vodíkových iontů MeSH
- lidé MeSH
- odchylka pozorovatele MeSH
- porodnictví statistika a číselné údaje MeSH
- reprodukovatelnost výsledků MeSH
- senzitivita a specificita MeSH
- software MeSH
- Check Tag
- lidé MeSH
- Publikační typ
- časopisecké články MeSH
- práce podpořená grantem MeSH
Staging criteria for renal cell carcinoma differ from many other cancers, in that renal tumors are often spherical with subtle, finger-like extensions into veins, renal sinus, or perinephric tissue. We sought to study interobserver agreement in pathologic stage categories for challenging cases. An online survey was circulated to urologic pathologists interested in kidney tumors, yielding 89% response (31/35). Most questions included 1 to 4 images, focusing on: vascular and renal sinus invasion (n=24), perinephric invasion (n=9), and gross pathology/specimen handling (n=17). Responses were collapsed for analysis into positive and negative/equivocal for upstaging. Consensus was regarded as an agreement of 67% (2/3) of participants, which was reached in 20/33 (61%) evaluable scenarios regarding renal sinus, perinephric, or vein invasion, of which 13/33 (39%) had ≥80% consensus. Lack of agreement was especially encountered regarding small tumor protrusions into a possible vascular lumen, close to the tumor leading edge. For gross photographs, most were interpreted as suspicious but requiring histologic confirmation. Most participants (61%) rarely used special stains to evaluate vascular invasion, usually endothelial markers (81%). Most agreed that a spherical mass bulging well beyond the kidney parenchyma into the renal sinus (71%) or perinephric fat (90%) did not necessarily indicate invasion. Interobserver agreement in pathologic staging of renal cancer is relatively good among urologic pathologists interested in kidney tumors, even when selecting cases that test the earliest and borderline thresholds for extrarenal extension. Disagreements remain, however, particularly for tumors with small, finger-like protrusions, closely juxtaposed to the main mass.
- MeSH
- karcinom z renálních buněk patologie MeSH
- laboratorní medicína metody MeSH
- lidé MeSH
- nádory ledvin patologie MeSH
- odchylka pozorovatele MeSH
- patologové MeSH
- staging nádorů metody MeSH
- urologie metody MeSH
- Check Tag
- lidé MeSH
- Publikační typ
- časopisecké články MeSH
Polymorphous adenocarcinoma (PAC) shows histologic diversity with streaming and targetoid features whereas cribriform adenocarcinoma of salivary gland (CASG) demonstrates predominantly cribriform and solid patterns with glomeruloid structures and optically clear nuclei. Opinions diverge on whether CASG represents a separate entity or a variant of PAC. We aimed to assess the level of agreement among 25 expert Head and Neck pathologists in classifying these tumors. Digital slides of 48 cases were reviewed and classified as: PAC, CASG, tumors with ≥50% of papillary architecture (PAP), and tumors with indeterminate features (IND). The consensus diagnoses were correlated with a previously reported molecular alteration. The consensus diagnoses were PAC in 18/48, CASG in16/48, PAP in 3/48, and IND in 11/48. There was a fair interobserver agreement in classifying the tumors (κ=0.370). The full consensus was achieved in 3 (6%) cases, all of which were classified as PAC. A moderate agreement was reached for PAC (κ=0.504) and PAP (κ=0.561), and a fair agreement was reached for CASG (κ=0.390). IND had only slight diagnostic concordance (κ=0.091). PAC predominantly harbored PRKD1 hotspot mutation, whereas CASG was associated with fusion involving PRKD1, PRKD2, or PRKD3. However, such molecular events were not exclusive as 7% of PAC had fusion and 13% of CASG had mutation. In conclusion, a fair to moderate interobserver agreement can be achieved in classifying PAC and CASG. However, a subset (23%) showed indeterminate features and was difficult to place along the morphologic spectrum of PAC/CASG among expert pathologists. This may explain the controversy in classifying these tumors.
- MeSH
- adenokarcinom klasifikace genetika patologie MeSH
- biopsie MeSH
- fúze genů MeSH
- genetická predispozice k nemoci MeSH
- hybridizace in situ fluorescenční MeSH
- kvantitativní polymerázová řetězová reakce MeSH
- lidé MeSH
- mutace MeSH
- mutační analýza DNA MeSH
- nádorové biomarkery genetika MeSH
- nádory slinných žláz klasifikace genetika patologie MeSH
- odchylka pozorovatele MeSH
- prediktivní hodnota testů MeSH
- reprodukovatelnost výsledků MeSH
- Check Tag
- lidé MeSH
- Publikační typ
- časopisecké články MeSH
- multicentrická studie MeSH
- Research Support, N.I.H., Extramural MeSH
- Geografické názvy
- Evropa MeSH
- Kanada MeSH
- Spojené státy americké MeSH
- Názvy látek
- nádorové biomarkery MeSH
BACKGROUND: Determining capture type and septal lead location during left bundle branch area pacing (LBBAP) relies on criteria obtained during implantation. However, during follow-up, the interpretation of left bundle branch (LBB) capture largely depends on QRS morphology, which is not so straightforward in LBBAP. OBJECTIVE: This study aimed to investigate the inter- and intraobserver agreement, as well as the accuracy of clinical judgment of the electrocardiogram (ECG) in determining LBB-capture and septal lead position in patients undergoing LBBAP implantation. In addition, the role of vectorcardiographic QRS-area in determining LBB-capture was evaluated. METHODS: Unipolar paced ECGs during LBBAP implantation from 50 patients with baseline narrow QRS were collected. LBB-capture was attempted in all patients and assessed using MELOS (Multicentre European Left Bundle Branch Area Pacing Outcomes Study) criteria and the European Heart Rhythm Association (EHRA) consensus statement. Eight blinded cardiologists classified 100 ECGs for capture type and septal location. RESULTS: The interobserver and intraobserver agreement for capture type had a Light's kappa of 0.43 and 0.62, respectively. Concordance between clinical judgment and intraprocedural confirmation averaged 72%. Interobserver and intraobserver agreement for septal lead position had a Light's kappa of 0.43 and 0.77 respectively. QRS-area was significantly higher for left ventricular septal pacing (LVSP) than nsLBBP, whereas QRS duration was not. A QRS-area cutoff of 26 mV.ms had 77% accuracy in distinguishing LVSP from nsLBBP. Clinical judgment accuracy averaged 72%. CONCLUSION: Interobserver agreement and correlation with intraprocedural confirmation (gold standard) are only moderate, whereas intraobserver agreement on ECG-based differentiation of capture type and septal lead location is substantial. Vectorcardiographic QRS-area slightly outperforms clinical judgment in distinguishing capture types and may be a useful objective alternative.
- Klíčová slova
- Capture type, Conduction system pacing, Interobserver and intraobserver agreement, Left bundle branch area pacing, Vectorcardiography,
- Publikační typ
- časopisecké články MeSH
OBJECTIVES: The aim is to estimate agreement between two-dimensional transvaginal ultrasound (2D-TVS) and three-dimensional volume contrast imaging (3D-VCI) in diagnosing deep myometrial invasion (MI) and cervical stromal involvement (CSI) of endometrial cancer and to compare the two methods regarding inter-rater reliability and diagnostic accuracy. METHODS: Fifteen ultrasound experts assessed off-line de-identified 3D-VCI volumes and 2D-TVU video clips from 58 patients with biopsy-confirmed endometrial cancer regarding the presence of deep (≥50%) MI and CSI. Video clips and 3D volumes were assessed independently. Interrater reliability was measured using kappa statistics. Histological diagnosis after hysterectomy served as gold standard. Accuracy measurements were correlated to rater experience using Spearman's rank correlation coefficient (ρ). RESULTS: Agreement between 2D-TVU and 3D-VCI for diagnosing MI was median 76% (range 64-93%) and for CSI median 88% (range 79-97%). Interrater reliability was better for 2D-TVU than for 3D-VCI (Fleiss' kappa 0.41 vs. 0.31 for MI and 0.55 vs. 0.45 for CSI). Median accuracy for diagnosing deep MI was 76% (range 59-84%) with 2D-TVU and 69% (range 52-83%) for 3D-VCI; the corresponding figures for CSI were 88% (range 81-93%) and 86% (range 72-95%). Accuracy was significantly correlated to how many cases the raters assessed annually. CONCLUSIONS: Off-line assessment of MI and CSI in women with endometrial cancer using 3D-VCI has lower interrater reliability and lower accuracy than 2D-TVU video clip assessment. Since accuracy was correlated to the number of cases assessed annually it is advised to centralize these examinations to high-volume centres.
- Klíčová slova
- Diagnostic imaging, Endometrial neoplasms, Neoplasm staging, Reproducibility of results, Three-dimensional imaging, Ultrasonography,
- MeSH
- dospělí MeSH
- invazivní růst nádoru MeSH
- kontrastní látky MeSH
- lidé středního věku MeSH
- lidé MeSH
- myometrium diagnostické zobrazování patologie MeSH
- nádory endometria diagnostické zobrazování patologie chirurgie MeSH
- odchylka pozorovatele MeSH
- reprodukovatelnost výsledků MeSH
- retrospektivní studie MeSH
- senioři nad 80 let MeSH
- senioři MeSH
- staging nádorů metody MeSH
- ultrasonografie metody MeSH
- zobrazování trojrozměrné * MeSH
- Check Tag
- dospělí MeSH
- lidé středního věku MeSH
- lidé MeSH
- senioři nad 80 let MeSH
- senioři MeSH
- ženské pohlaví MeSH
- Publikační typ
- časopisecké články MeSH
- práce podpořená grantem MeSH
- srovnávací studie MeSH
- Názvy látek
- kontrastní látky MeSH
OBJECTIVES: The main objective was to generate a GLobal OMERACT Ultrasound DActylitis Score (GLOUDAS) in psoriatic arthritis and to test its reliability. To this end, we assessed the validity, feasibility and applicability of ultrasound assessment of finger entheses to incorporate them into the scoring system. METHODS: The study consisted of a stepwise process. First, in cadaveric specimens, we identified enthesis sites of the fingers by ultrasound and gross anatomy, and then verified presence of entheseal tissue in histological samples. We then selected the entheses to be incorporated into a dactylitis scoring system through a Delphi consensus process among international experts. Next, we established and defined the ultrasound components of dactylitis and their scoring systems using Delphi methodology. Finally, we tested the interobserver and intraobserver reliability of the consensus- based scoring systemin patients with psoriatic dactylitis. RESULTS: 32 entheses were identified in cadaveric fingers. The presence of entheseal tissues was confirmed in all cadaveric samples. Of these, following the consensus process, 12 entheses were selected for inclusion in GLOUDAS. Ultrasound components of GLOUDAS agreed on through the Delphi process were synovitis, tenosynovitis, enthesitis, subcutaneous tissue inflammation and periextensor tendon inflammation. The scoring system for each component was also agreed on. Interobserver reliability was fair to good (κ 0.39-0.71) and intraobserver reliability good to excellent (κ 0.80-0.88) for dactylitis components. Interobserver and intraobserver agreement for the total B-mode and Doppler mode scores (sum of the scores of the individual abnormalities) were excellent (interobserver intraclass correlation coefficient (ICC) 0.98 for B-mode and 0.99 for Doppler mode; intraobserver ICC 0.98 for both modes). CONCLUSIONS: We have produced a consensus-driven ultrasound dactylitis scoring system that has shown acceptable interobserver reliability and excellent intraobserver reliability. Through anatomical knowledge, small entheses of the fingers were identified and histologically validated.
- Klíčová slova
- Arthritis, Psoriatic, Inflammation, Synovitis, Ultrasonography,
- MeSH
- delfská metoda MeSH
- dospělí MeSH
- entezopatie diagnostické zobrazování MeSH
- kloub prstu ruky * diagnostické zobrazování patologie MeSH
- lidé středního věku MeSH
- lidé MeSH
- mrtvola MeSH
- odchylka pozorovatele MeSH
- prsty ruky diagnostické zobrazování patologie MeSH
- psoriatická artritida * diagnostické zobrazování MeSH
- reprodukovatelnost výsledků MeSH
- senioři MeSH
- studie proveditelnosti MeSH
- stupeň závažnosti nemoci * MeSH
- synovitida diagnostické zobrazování patologie MeSH
- tenosynovitida diagnostické zobrazování MeSH
- ultrasonografie * metody MeSH
- Check Tag
- dospělí MeSH
- lidé středního věku MeSH
- lidé MeSH
- mužské pohlaví MeSH
- senioři MeSH
- ženské pohlaví MeSH
- Publikační typ
- časopisecké články MeSH
- validační studie MeSH
BACKGROUND: Mucosal visualization during upper gastrointestinal (UGI) endoscopy can be impaired by the presence of foam, bubbles, and mucus. Some UGI endoscopy visibility scales have been proposed but have not undergone multicenter validation. This study aimed to develop and validate the Gastroscopy RAte of Cleanliness Evaluation (GRACE) scale. METHODS: A multicenter, international, cross-sectional study was conducted. The GRACE scale is based on a score from 0 (worst) to 3 (excellent) for esophagus, stomach, and duodenum, for a total ranging from 0 to 9. In phase 1, four expert endoscopists evaluated 60 images twice, with a 2-week interval between rounds; in phase 2, the same 60 images were scored twice by one expert and one nonexpert endoscopist from 27 endoscopy departments worldwide. For reproducibility assessment and real-time validation, the scale was applied to consecutive patients undergoing gastroscopy at each center. RESULTS: On internal validation, interobserver agreement was 0.81 (95 %CI 0.73-0.87) and 0.80 (95 %CI 0.72-0.86), with reliability of 0.73 (95 %CI 0.63-0.82) and 0.72 (95 %CI 0.63-0.81), in the two rounds, respectively. On external validation, overall interobserver agreement was 0.85 (95 %CI 0.82-0.88) and reliability was 0.79 (95 %CI 0.73-0.84). In real-time evaluation, the overall proportion of correct classifications was 0.80 (95 %CI 0.77-0.82). CONCLUSIONS: The GRACE scale showed good interobserver agreement, reliability, and validity. The widespread use of this scale could enhance quality and standardize the assessment of mucosal cleanliness during UGI endoscopy, pushing endoscopists to strive for excellent visibility and reducing the risk of missed lesions.
- MeSH
- dospělí MeSH
- ezofágus diagnostické zobrazování MeSH
- gastroskopie * normy MeSH
- lidé středního věku MeSH
- lidé MeSH
- odchylka pozorovatele MeSH
- průřezové studie MeSH
- reprodukovatelnost výsledků MeSH
- senioři MeSH
- žaludek diagnostické zobrazování MeSH
- Check Tag
- dospělí MeSH
- lidé středního věku MeSH
- lidé MeSH
- mužské pohlaví MeSH
- senioři MeSH
- ženské pohlaví MeSH
- Publikační typ
- časopisecké články MeSH
- multicentrická studie MeSH
- validační studie MeSH