Benchmarks
Dotaz
Zobrazit nápovědu
elektronický časopis
- MeSH
- benchmarking MeSH
- ekonomika a organizace zdravotní péče MeSH
- zajištění kvality zdravotní péče organizace a řízení MeSH
- Konspekt
- Veřejné zdraví a hygiena
- NLK Obory
- management, organizace a řízení zdravotnictví
- NLK Publikační typ
- elektronické časopisy
Similar to the medical imaging community, the bioimaging community has recently realized the need to benchmark various image analysis methods to compare their performance and assess their suitability for specific applications. Challenges sponsored by prestigious conferences have proven to be an effective means of encouraging benchmarking and new algorithm development for a particular type of image data. Bioimage analysis challenges have recently complemented medical image analysis challenges, especially in the case of the International Symposium on Biomedical Imaging (ISBI). This review summarizes recent progress in this respect and describes the general process of designing a bioimage analysis benchmark or challenge, including the proper selection of datasets and evaluation metrics. It also presents examples of specific target applications and biological research tasks that have benefited from these challenges with respect to the performance of automatic image analysis methods that are crucial for the given task. Finally, available benchmarks and challenges in terms of common features, possible classification and implications drawn from the results are analysed.
- MeSH
- algoritmy MeSH
- benchmarking * MeSH
- databáze faktografické MeSH
- fluorescenční mikroskopie přístrojové vybavení metody normy MeSH
- lidé MeSH
- molekulární zobrazování přístrojové vybavení metody normy MeSH
- počítačové zpracování obrazu metody statistika a číselné údaje MeSH
- rozpoznávání automatizované statistika a číselné údaje MeSH
- Check Tag
- lidé MeSH
- Publikační typ
- časopisecké články MeSH
- práce podpořená grantem MeSH
- přehledy MeSH
BACKGROUND: Benchmark comparisons in surgery allow identification of gaps in the quality of care provided. The aim of this study was to determine quality thresholds for high (HAR) and low (LAR) anterior resections in colorectal cancer surgery by applying the concept of benchmarking. METHODS: This 5-year multinational retrospective study included patients who underwent anterior resection for cancer in 19 high-volume centres on five continents. Benchmarks were defined for 11 relevant postoperative variables at discharge, 3 months, and 6 months (for LAR). Benchmarks were calculated for two separate cohorts: patients without (ideal) and those with (non-ideal) outcome-relevant co-morbidities. Benchmark cut-offs were defined as the 75th percentile of each centre's median value. RESULTS: A total of 3903 patients who underwent HAR and 3726 who had LAR for cancer were analysed. After 3 months' follow-up, the mortality benchmark in HAR for ideal and non-ideal patients was 0.0 versus 3.0 per cent, and in LAR it was 0.0 versus 2.2 per cent. Benchmark results for anastomotic leakage were 5.0 versus 6.9 per cent for HAR, and 13.6 versus 11.8 per cent for LAR. The overall morbidity benchmark in HAR was a Comprehensive Complication Index (CCI®) score of 8.6 versus 14.7, and that for LAR was CCI® score 11.9 versus 18.3. CONCLUSION: Regular comparison of individual-surgeon or -unit outcome data against benchmark thresholds may identify gaps in care quality that can improve patient outcome.
- MeSH
- benchmarking MeSH
- kolorektální chirurgie * MeSH
- lidé MeSH
- nádory rekta * chirurgie MeSH
- pooperační komplikace epidemiologie etiologie MeSH
- proktektomie * MeSH
- retrospektivní studie MeSH
- Check Tag
- lidé MeSH
- Publikační typ
- časopisecké články MeSH
The aim was to provide benchmarks and investigate contribution of start and turn performances in the European Short-Course Swimming Championship. Over all race distances, 932 individual races of male competitors were video captured and the start and turn performances were analysed. Start and turn performances contributed up to 26% and 56% of the total race time. Analysis of variance showed that the 15 m start times were faster for Freestyle and Butterfly (p < 0.001) compared to the other swimming strokes. In-water starts (Backstroke) were slower at the 5 m mark compare to on-block starts (p < 0.001). Tumble turns were faster than open turns (p < 0.001). Multiple linear regression analysis explained 82-97% of total variance in the race results with a decreasing effect of start performance for the longer distance races. Turn performance affected the results across all race distances (p < 0.001). The benchmarks and percentiles provide comparative data for swimmers of different performance levels. Considering the large contribution of start and turn performance to race time and the high effect in the regression model, training regimes that are mainly based on conditioning of free-swimming should be reconsidered.
- MeSH
- benchmarking * MeSH
- biomechanika MeSH
- kompetitivní chování MeSH
- lidé MeSH
- plavání MeSH
- sportovní výkon * MeSH
- Check Tag
- lidé MeSH
- mužské pohlaví MeSH
- Publikační typ
- časopisecké články MeSH
BACKGROUND: Recently, deep neural networks have been successfully applied in many biological fields. In 2020, a deep learning model AlphaFold won the protein folding competition with predicted structures within the error tolerance of experimental methods. However, this solution to the most prominent bioinformatic challenge of the past 50 years has been possible only thanks to a carefully curated benchmark of experimentally predicted protein structures. In Genomics, we have similar challenges (annotation of genomes and identification of functional elements) but currently, we lack benchmarks similar to protein folding competition. RESULTS: Here we present a collection of curated and easily accessible sequence classification datasets in the field of genomics. The proposed collection is based on a combination of novel datasets constructed from the mining of publicly available databases and existing datasets obtained from published articles. The collection currently contains nine datasets that focus on regulatory elements (promoters, enhancers, open chromatin region) from three model organisms: human, mouse, and roundworm. A simple convolution neural network is also included in a repository and can be used as a baseline model. Benchmarks and the baseline model are distributed as the Python package 'genomic-benchmarks', and the code is available at https://github.com/ML-Bioinfo-CEITEC/genomic_benchmarks . CONCLUSIONS: Deep learning techniques revolutionized many biological fields but mainly thanks to the carefully curated benchmarks. For the field of Genomics, we propose a collection of benchmark datasets for the classification of genomic sequences with an interface for the most commonly used deep learning libraries, implementation of the simple neural network and a training framework that can be used as a starting point for future research. The main aim of this effort is to create a repository for shared datasets that will make machine learning for genomics more comparable and reproducible while reducing the overhead of researchers who want to enter the field, leading to healthy competition and new discoveries.
Forests are increasingly affected by natural disturbances. Subsequent salvage logging, a widespread management practice conducted predominantly to recover economic capital, produces further disturbance and impacts biodiversity worldwide. Hence, naturally disturbed forests are among the most threatened habitats in the world, with consequences for their associated biodiversity. However, there are no evidence-based benchmarks for the proportion of area of naturally disturbed forests to be excluded from salvage logging to conserve biodiversity. We apply a mixed rarefaction/extrapolation approach to a global multi-taxa dataset from disturbed forests, including birds, plants, insects and fungi, to close this gap. We find that 75 ± 7% (mean ± SD) of a naturally disturbed area of a forest needs to be left unlogged to maintain 90% richness of its unique species, whereas retaining 50% of a naturally disturbed forest unlogged maintains 73 ± 12% of its unique species richness. These values do not change with the time elapsed since disturbance but vary considerably among taxonomic groups.
- MeSH
- benchmarking MeSH
- biodiverzita MeSH
- druhová specificita MeSH
- ekosystém MeSH
- lesnictví normy MeSH
- lesy * MeSH
- zachování přírodních zdrojů * metody MeSH
- zvířata MeSH
- Check Tag
- zvířata MeSH
- Publikační typ
- časopisecké články MeSH
- práce podpořená grantem MeSH
Computational models of acoustic wave propagation are frequently used in transcranial ultrasound therapy, for example, to calculate the intracranial pressure field or to calculate phase delays to correct for skull distortions. To allow intercomparison between the different modeling tools and techniques used by the community, an international working group was convened to formulate a set of numerical benchmarks. Here, these benchmarks are presented, along with intercomparison results. Nine different benchmarks of increasing geometric complexity are defined. These include a single-layer planar bone immersed in water, a multi-layer bone, and a whole skull. Two transducer configurations are considered (a focused bowl and a plane piston operating at 500 kHz), giving a total of 18 permutations of the benchmarks. Eleven different modeling tools are used to compute the benchmark results. The models span a wide range of numerical techniques, including the finite-difference time-domain method, angular spectrum method, pseudospectral method, boundary-element method, and spectral-element method. Good agreement is found between the models, particularly for the position, size, and magnitude of the acoustic focus within the skull. When comparing results for each model with every other model in a cross-comparison, the median values for each benchmark for the difference in focal pressure and position are less than 10% and 1 mm, respectively. The benchmark definitions, model results, and intercomparison codes are freely available to facilitate further comparisons.
MOTIVATION: Automatic tracking of cells in multidimensional time-lapse fluorescence microscopy is an important task in many biomedical applications. A novel framework for objective evaluation of cell tracking algorithms has been established under the auspices of the IEEE International Symposium on Biomedical Imaging 2013 Cell Tracking Challenge. In this article, we present the logistics, datasets, methods and results of the challenge and lay down the principles for future uses of this benchmark. RESULTS: The main contributions of the challenge include the creation of a comprehensive video dataset repository and the definition of objective measures for comparison and ranking of the algorithms. With this benchmark, six algorithms covering a variety of segmentation and tracking paradigms have been compared and ranked based on their performance on both synthetic and real datasets. Given the diversity of the datasets, we do not declare a single winner of the challenge. Instead, we present and discuss the results for each individual dataset separately. AVAILABILITY AND IMPLEMENTATION: The challenge Web site (http://www.codesolorzano.com/celltrackingchallenge) provides access to the training and competition datasets, along with the ground truth of the training videos. It also provides access to Windows and Linux executable files of the evaluation software and most of the algorithms that competed in the challenge.
The Cell Tracking Challenge is an ongoing benchmarking initiative that has become a reference in cell segmentation and tracking algorithm development. Here, we present a significant number of improvements introduced in the challenge since our 2017 report. These include the creation of a new segmentation-only benchmark, the enrichment of the dataset repository with new datasets that increase its diversity and complexity, and the creation of a silver standard reference corpus based on the most competitive results, which will be of particular interest for data-hungry deep learning-based strategies. Furthermore, we present the up-to-date cell segmentation and tracking leaderboards, an in-depth analysis of the relationship between the performance of the state-of-the-art methods and the properties of the datasets and annotations, and two novel, insightful studies about the generalizability and the reusability of top-performing methods. These studies provide critical practical conclusions for both developers and users of traditional and machine learning-based cell segmentation and tracking algorithms.
In this critical review, we comment on the absence of widely shared benchmark problems and relevant challenges or even attractive competitions in the field of electrokinetics. We argue that in some other scientific domains that are, similarly as electrokinetics, strongly multidisciplinary, the existence of these tools is very beneficial because it stimulates the discussion about what constitutes the bottleneck of further progress, allows easier exploitation of results provided by other scientific and engineering disciplines, and, last but not least, makes the research domain attractive and visible to a broader public, including students. The goal of this review is to provoke some discussion that might perhaps lead to compensating for these shortcomings.
