The Cell Tracking Challenge is an ongoing benchmarking initiative that has become a reference in cell segmentation and tracking algorithm development. Here, we present a significant number of improvements introduced in the challenge since our 2017 report. These include the creation of a new segmentation-only benchmark, the enrichment of the dataset repository with new datasets that increase its diversity and complexity, and the creation of a silver standard reference corpus based on the most competitive results, which will be of particular interest for data-hungry deep learning-based strategies. Furthermore, we present the up-to-date cell segmentation and tracking leaderboards, an in-depth analysis of the relationship between the performance of the state-of-the-art methods and the properties of the datasets and annotations, and two novel, insightful studies about the generalizability and the reusability of top-performing methods. These studies provide critical practical conclusions for both developers and users of traditional and machine learning-based cell segmentation and tracking algorithms.

Bioengineering Department Universidad Carlos 3 de Madrid Madrid Spain

Biomedical Engineering Program and Ciberonc Center for Applied Medical Research Universidad de Navarra Pamplona Spain

Boston Children's Hospital and Harvard Medical School Boston MA USA

Center for Advanced Methods in Biological Image Analysis Beckman Institute California Institute of Technology Pasadena CA USA

Centre for Biomedical Image Analysis Faculty of Informatics Masaryk University Brno Czech Republic

Centre National de la Recherche Scientifique Paris France

Centro de Informatica Universidade Federal de Pernambuco Recife Brazil

CIVA Lab Department of Electrical Engineering and Computer Science University of Missouri Columbia MO USA

Department of Electrical and Computer Engineering Drexel University Philadelphia PA USA

Division of Biology and Biological Engineering and Howard Hughes Medical Institute California Institute of Technology Pasadena CA USA

Division of Medical Image Computing German Cancer Research Center Heidelberg Germany

Griffith University Nathan Queensland Australia

Helmholtz Imaging German Cancer Research Center Heidelberg Germany

Institut de Génomique Fonctionnelle de Lyon École Normale Supérieure de Lyon Lyon France

Institute for Automation and Applied Informatics Karlsruhe Institute of Technology Eggenstein Leopoldshafen Germany

Instituto de Investigación Sanitaria Gregorio Marañón Madrid Spain

Interactive Machine Learning Group German Cancer Research Center Heidelberg Germany

IT4Innovations National Supercomputing Center VSB Technical University of Ostrava Ostrava Czech Republic

Optical Cell Biology Instituto Gulbenkian de Ciência Oeiras Portugal

Pattern Analysis and Learning Group Department of Radiation Oncology Heidelberg University Hospital Heidelberg Germany

Raysearch Laboratories AB Stockholm Sweden

School of Computer Science and Engineering University of New South Wales Sydney New South Wales Australia

School of Electrical and Computer Engineering Ben Gurion University of the Negev Beersheba Israel

The Elmore Family School of Electrical and Computer Engineering Purdue University West Lafayette IN USA

Zobrazit více v PubMed

May M, Denecke B, Schroeder T, Götz M, Faissner A. Cell tracking in vitro reveals that the extracellular matrix glycoprotein Tenascin-C modulates cell cycle length and differentiation in neural stem/progenitor cells of the developing mouse spinal cord. Biol. Open. 2018;7:bio027730. doi: 10.1242/bio.027730. PubMed DOI PMC

Kazwiny Y, et al. Extracting neuronal activity signals from microscopy recordings of contractile tissue using B-spline explicit active surfaces (BEAS) cell tracking. Sci. Rep. 2021;11:10937. doi: 10.1038/s41598-021-90448-4. PubMed DOI PMC

Lovas JR, Yuste R. Ensemble synchronization in the reassembly of Hydra’s nervous system. Curr. Biol. 2022;31:3784–3796. doi: 10.1016/j.cub.2021.06.047. PubMed DOI

Schermelleh L, et al. Super-resolution microscopy demystified. Nat. Cell Biol. 2019;21:72–84. doi: 10.1038/s41556-018-0251-8. PubMed DOI

Lewis SM, et al. Spatial omics and multiplexed imaging to explore cancer biology. Nat. Methods. 2021;18:997–1012. doi: 10.1038/s41592-021-01203-6. PubMed DOI

Girkin JM, Carvalho MT. The light-sheet microscopy revolution. J. Optics. 2018;20:053002. doi: 10.1088/2040-8986/aab58a. DOI

Meijering E. A bird’s-eye view of deep learning in bioimage analysis. Comput. Struct. Biotechnol. J. 2020;18:2312–2325. doi: 10.1016/j.csbj.2020.08.003. PubMed DOI PMC

Maška M, et al. A benchmark for comparison of cell tracking algorithms. Bioinformatics. 2014;30:1609–1617. doi: 10.1093/bioinformatics/btu080. PubMed DOI PMC

Ulman V, et al. An objective comparison of cell-tracking algorithms. Nat. Methods. 2017;14:1141–1152. doi: 10.1038/nmeth.4473. PubMed DOI PMC

Ronneberger, O., Fischer, P. & Brox, T. U-net: convolutional networks for biomedical image segmentation. In

Svoboda D, Ulman V. MitoGen: a framework for generating 3D synthetic time-lapse sequences of cell populations in fluorescence microscopy. IEEE Trans. Med. Imaging. 2017;36:310–321. doi: 10.1109/TMI.2016.2606545. PubMed DOI

Sorokin DV, et al. FiloGen: a model-based generator of synthetic 3-D time-lapse sequences of single motile cells with growing and branching filopodia. IEEE Trans. Med. Imaging. 2018;37:2630–2641. doi: 10.1109/TMI.2018.2845884. PubMed DOI

Guerrero Peña, F. A. et al. J-regularization improves imbalanced multiclass segmentation. In

Scherr T, Löffler K, Böhland M, Mikut R. Cell segmentation and tracking using CNN-based distance predictions and a graph-based matching strategy. PLoS One. 2020;15:e0243219. doi: 10.1371/journal.pone.0243219. PubMed DOI PMC

Isensee F, Jaeger PF, Kohl SAA, Petersen J, Maier-Hein KH. nnU-Net: a self-configuring method for deep learning based biomedical image segmentation. Nat. Methods. 2021;18:203–211. doi: 10.1038/s41592-020-01008-z. PubMed DOI

Löffler K, Mikut R. EmbedTrack: simultaneous cell segmentation and tracking through learning offsets and clustering bandwidths. IEEE Access. 2022;10:77147–77157. doi: 10.1109/ACCESS.2022.3192880. DOI

Magnusson KEG, Jaldén J, Gilbert PM, Blau HM. Global linking of cell tracks using the Viterbi algorithm. IEEE Trans. Med. Imaging. 2015;34:1–19. doi: 10.1109/TMI.2014.2370951. PubMed DOI PMC

Guo, T., Wang, Y., Solorio, L. & Allebach, J. P. Training a universal instance segmentation network for live cell images of various cell types and imaging modalities. Preprint at 10.48550/arxiv.2207.14347 (2022).

Arbelle A, Cohen S, Riklin Raviv T. Dual-task ConvLSTM-UNet for instance segmentation of weakly annotated microscopy videos. IEEE Trans. Med. Imaging. 2022;41:1948–1960. doi: 10.1109/TMI.2022.3152927. PubMed DOI

Ben-Haim, T & Riklin Raviv, T. Graph neural network for cell tracking in microscopy videos. In

Sugawara K, Çevrim Ç, Averof M. Tracking cell lineages in 3D by incremental deep learning. eLife. 2022;11:e69380. doi: 10.7554/eLife.69380. PubMed DOI PMC

Lux, F. & Matula, P. DIC image segmentation of dense cell populations by combining deep learning and watershed. In

Bao, R., Al-Shakarji, N. M., Bunyak, F. & Palaniappan, K. DMNet: dual-stream marker guided deep network for dense cell segmentation and lineage tracking. In PubMed PMC

Rahmon, G., Bunyak, F. & Palaniappan, K. Motion U-Net: multi-cue encoder–decoder network for motion segmentation. In

Girshick, R., Donahue, J., Darrell, T. & Malik, J. Rich feature hierarchies for accurate object detection and semantic segmentation. In

Wang J, et al. Deep high-resolution representation learning for visual recognition. IEEE Trans. Pattern Anal. Mach. Intell. 2021;43:3349–3364. doi: 10.1109/TPAMI.2020.2983686. PubMed DOI

Malin-Mayor C, et al. Automated reconstruction of whole-embryo lineages by learning from sparse annotations. Nat. Biotechnol. 2023;41:44–49. doi: 10.1038/s41587-022-01427-7. PubMed DOI PMC

Horn BK, Schunck BG. Determining optical flow. Artif. Intell. 1981;17:185–203. doi: 10.1016/0004-3702(81)90024-2. DOI

Dosovitskiy, A. et al. Flownet: learning optical flow with convolutional networks. In

Ranjan, A. & Black, M. J. Optical flow estimation using a spatial pyramid network. In

Sun, D., Yang, X., Liu, M. Y. & Kautz, J. Pwc-net: CNNs for optical flow using pyramid, warping, and cost volume. In

Teed, Z. & Deng, J. Raft: recurrent all-pairs field transforms for optical flow. In

Chan, K. C., Wang, X., Yu, K., Dong, C. & Loy, C. C. BasicVSR: the search for essential components in video super-resolution and beyond. In

Niklaus, S., Hu, P. & Chen, J. Splatting-based synthesis for video frame interpolation. In

Osokin, A., Chessel, A., Carazo Salas, R. E. & Vaggi, F. GANs for biological image synthesis. In

Magnusson, K. E. G. & Jaldén, J. Tracking of non-Brownian particles using the Viterbi algorithm. In

Arzt M, et al. LABKIT: labeling and segmentation toolkit for big image data. Front. Comput. Sci. 2022;4:777728. doi: 10.3389/fcomp.2022.777728. DOI

Akbaş, C. E., Ulman, V., Maška, M., Jug, F. & Kozubek, M. Automatic fusion of segmentation and tracking labels. In

Matula P, et al. Cell tracking accuracy measurement based on comparison of acyclic oriented graphs. PLoS One. 2015;10:e0144959. doi: 10.1371/journal.pone.0144959. PubMed DOI PMC

Guerrero Peña, F. A. et al. Multiclass weighted loss for instance segmentation of cluttered cells. In

Guerrero Peña, F. A., Fernandez, P. D. M., Ren, T. I. & Cunha A. A weakly supervised method for instance segmentation of biological cells. In

Padfield D, Rittscher J, Roysam B. Coupled minimum-cost flow cell tracking for high-throughput quantitative analysis. Med. Image Anal. 2011;15:650–668. doi: 10.1016/j.media.2010.07.006. PubMed DOI

Löffler K, Scherr T, Mikut R. A graph-based cell tracking algorithm with few manually tunable parameters and automated segmentation error correction. PLoS One. 2021;16:e0249257. doi: 10.1371/journal.pone.0249257. PubMed DOI PMC

Antonelli M, et al. The medical segmentation decathlon. Nature Commun. 2022;12:4128. doi: 10.1038/s41467-022-30695-9. PubMed DOI PMC

Neven, D., Brabandere, B. D., Proesmans, M. & Van Gool, L. Instance segmentation by jointly optimizing spatial embeddings and clustering bandwidth. In

Romera E, Álvarez JM, Bergasa LM, Arroyo R. ERFNet: efficient residual factorized ConvNet for real-time semantic segmentation. IEEE Trans. Intell. Transport. Syst. 2018;19:263–272. doi: 10.1109/TITS.2017.2750080. DOI

R Core Team.

MIFA: Metadata, Incentives, Formats and Accessibility guidelines to improve the reuse of AI datasets for bioimage analysis

Comparison of Independent Analyses of Identical Image Sets Reveals Significant Analyst-to-Analyst Variability

Topology-preserving contourwise shape fusion

Comparing Deep Learning Performance for Chronic Lymphocytic Leukaemia Cell Segmentation in Brightfield Microscopy Images