Computational competitions are the standard for benchmarking medical image analysis algorithms, but they typically use small curated test datasets acquired at a few centers, leaving a gap to the reality of diverse multicentric patient data. To this end, the Federated Tumor Segmentation (FeTS) Challenge represents the paradigm for real-world algorithmic performance evaluation. The FeTS challenge is a competition to benchmark (i) federated learning aggregation algorithms and (ii) state-of-the-art segmentation algorithms, across multiple international sites. Weight aggregation and client selection techniques were compared using a multicentric brain tumor dataset in realistic federated learning simulations, yielding benefits for adaptive weight aggregation, and efficiency gains through client sampling. Quantitative performance evaluation of state-of-the-art segmentation algorithms on data distributed internationally across 32 institutions yielded good generalization on average, albeit the worst-case performance revealed data-specific modes of failure. Similar multi-site setups can help validate the real-world utility of healthcare AI algorithms in the future.

• MeSH
• Algorithms * MeSH
• Benchmarking * methods   MeSH
• Humans MeSH
• Magnetic Resonance Imaging MeSH
• Brain Neoplasms * diagnostic imaging   MeSH
• Image Processing, Computer-Assisted * methods   MeSH
• Artificial Intelligence * MeSH
• Check Tag
• Humans MeSH
• Publication type
• Journal Article MeSH