The Cell Tracking Challenge is an ongoing benchmarking initiative that has become a reference in cell segmentation and tracking algorithm development. Here, we present a significant number of improvements introduced in the challenge since our 2017 report. These include the creation of a new segmentation-only benchmark, the enrichment of the dataset repository with new datasets that increase its diversity and complexity, and the creation of a silver standard reference corpus based on the most competitive results, which will be of particular interest for data-hungry deep learning-based strategies. Furthermore, we present the up-to-date cell segmentation and tracking leaderboards, an in-depth analysis of the relationship between the performance of the state-of-the-art methods and the properties of the datasets and annotations, and two novel, insightful studies about the generalizability and the reusability of top-performing methods. These studies provide critical practical conclusions for both developers and users of traditional and machine learning-based cell segmentation and tracking algorithms.

Bioengineering Department Universidad Carlos 3 de Madrid Madrid Spain

Biomedical Engineering Program and Ciberonc Center for Applied Medical Research Universidad de Navarra Pamplona Spain

Boston Children's Hospital and Harvard Medical School Boston MA USA

Center for Advanced Methods in Biological Image Analysis Beckman Institute California Institute of Technology Pasadena CA USA

Centre for Biomedical Image Analysis Faculty of Informatics Masaryk University Brno Czech Republic

Centre National de la Recherche Scientifique Paris France

Centro de Informatica Universidade Federal de Pernambuco Recife Brazil

CIVA Lab Department of Electrical Engineering and Computer Science University of Missouri Columbia MO USA

Department of Electrical and Computer Engineering Drexel University Philadelphia PA USA

Division of Biology and Biological Engineering and Howard Hughes Medical Institute California Institute of Technology Pasadena CA USA

Division of Medical Image Computing German Cancer Research Center Heidelberg Germany

Griffith University Nathan Queensland Australia

Helmholtz Imaging German Cancer Research Center Heidelberg Germany

Institut de Génomique Fonctionnelle de Lyon École Normale Supérieure de Lyon Lyon France

Institute for Automation and Applied Informatics Karlsruhe Institute of Technology Eggenstein Leopoldshafen Germany

Instituto de Investigación Sanitaria Gregorio Marañón Madrid Spain

Interactive Machine Learning Group German Cancer Research Center Heidelberg Germany

IT4Innovations National Supercomputing Center VSB Technical University of Ostrava Ostrava Czech Republic

Optical Cell Biology Instituto Gulbenkian de Ciência Oeiras Portugal

Pattern Analysis and Learning Group Department of Radiation Oncology Heidelberg University Hospital Heidelberg Germany

Raysearch Laboratories AB Stockholm Sweden

School of Computer Science and Engineering University of New South Wales Sydney New South Wales Australia

School of Electrical and Computer Engineering Ben Gurion University of the Negev Beersheba Israel

The Elmore Family School of Electrical and Computer Engineering Purdue University West Lafayette IN USA

Zobrazit více v PubMed

May M, Denecke B, Schroeder T, Götz M, Faissner A. Cell tracking in vitro reveals that the extracellular matrix glycoprotein Tenascin-C modulates cell cycle length and differentiation in neural stem/progenitor cells of the developing mouse spinal cord. Biol. Open. 2018;7:bio027730. doi: 10.1242/bio.027730. PubMed DOI PMC

Kazwiny Y, et al. Extracting neuronal activity signals from microscopy recordings of contractile tissue using B-spline explicit active surfaces (BEAS) cell tracking. Sci. Rep. 2021;11:10937. doi: 10.1038/s41598-021-90448-4. PubMed DOI PMC

Lovas JR, Yuste R. Ensemble synchronization in the reassembly of Hydra’s nervous system. Curr. Biol. 2022;31:3784–3796. doi: 10.1016/j.cub.2021.06.047. PubMed DOI

Schermelleh L, et al. Super-resolution microscopy demystified. Nat. Cell Biol. 2019;21:72–84. doi: 10.1038/s41556-018-0251-8. PubMed DOI

Lewis SM, et al. Spatial omics and multiplexed imaging to explore cancer biology. Nat. Methods. 2021;18:997–1012. doi: 10.1038/s41592-021-01203-6. PubMed DOI

Girkin JM, Carvalho MT. The light-sheet microscopy revolution. J. Optics. 2018;20:053002. doi: 10.1088/2040-8986/aab58a. DOI

Meijering E. A bird’s-eye view of deep learning in bioimage analysis. Comput. Struct. Biotechnol. J. 2020;18:2312–2325. doi: 10.1016/j.csbj.2020.08.003. PubMed DOI PMC

Maška M, et al. A benchmark for comparison of cell tracking algorithms. Bioinformatics. 2014;30:1609–1617. doi: 10.1093/bioinformatics/btu080. PubMed DOI PMC

Ulman V, et al. An objective comparison of cell-tracking algorithms. Nat. Methods. 2017;14:1141–1152. doi: 10.1038/nmeth.4473. PubMed DOI PMC

Ronneberger, O., Fischer, P. & Brox, T. U-net: convolutional networks for biomedical image segmentation. In Medical Image Computing and Computer-Assisted Intervention – MICCAI 2015. Lecture Notes in Computer Science Vol. 9351 (Eds. Navab, N. et al.) 234–241 (Springer, Cham, 2015).

Svoboda D, Ulman V. MitoGen: a framework for generating 3D synthetic time-lapse sequences of cell populations in fluorescence microscopy. IEEE Trans. Med. Imaging. 2017;36:310–321. doi: 10.1109/TMI.2016.2606545. PubMed DOI

Sorokin DV, et al. FiloGen: a model-based generator of synthetic 3-D time-lapse sequences of single motile cells with growing and branching filopodia. IEEE Trans. Med. Imaging. 2018;37:2630–2641. doi: 10.1109/TMI.2018.2845884. PubMed DOI

Guerrero Peña, F. A. et al. J-regularization improves imbalanced multiclass segmentation. In 2020 IEEE 17th International Symposium on Biomedical Imaging (ISBI), 1–5 (IEEE, 2020).

Scherr T, Löffler K, Böhland M, Mikut R. Cell segmentation and tracking using CNN-based distance predictions and a graph-based matching strategy. PLoS One. 2020;15:e0243219. doi: 10.1371/journal.pone.0243219. PubMed DOI PMC

Isensee F, Jaeger PF, Kohl SAA, Petersen J, Maier-Hein KH. nnU-Net: a self-configuring method for deep learning based biomedical image segmentation. Nat. Methods. 2021;18:203–211. doi: 10.1038/s41592-020-01008-z. PubMed DOI

Löffler K, Mikut R. EmbedTrack: simultaneous cell segmentation and tracking through learning offsets and clustering bandwidths. IEEE Access. 2022;10:77147–77157. doi: 10.1109/ACCESS.2022.3192880. DOI

Magnusson KEG, Jaldén J, Gilbert PM, Blau HM. Global linking of cell tracks using the Viterbi algorithm. IEEE Trans. Med. Imaging. 2015;34:1–19. doi: 10.1109/TMI.2014.2370951. PubMed DOI PMC

Guo, T., Wang, Y., Solorio, L. & Allebach, J. P. Training a universal instance segmentation network for live cell images of various cell types and imaging modalities. Preprint at 10.48550/arxiv.2207.14347 (2022).

Arbelle A, Cohen S, Riklin Raviv T. Dual-task ConvLSTM-UNet for instance segmentation of weakly annotated microscopy videos. IEEE Trans. Med. Imaging. 2022;41:1948–1960. doi: 10.1109/TMI.2022.3152927. PubMed DOI

Ben-Haim, T & Riklin Raviv, T. Graph neural network for cell tracking in microscopy videos. In Computer Vision – ECCV 2022. ECCV 2022. Lecture Notes in Computer Science Vol. 13681 (Eds. Avidan, S. et al.) (Springer, Cham, 2022)

Sugawara K, Çevrim Ç, Averof M. Tracking cell lineages in 3D by incremental deep learning. eLife. 2022;11:e69380. doi: 10.7554/eLife.69380. PubMed DOI PMC

Lux, F. & Matula, P. DIC image segmentation of dense cell populations by combining deep learning and watershed. In Proceedings of the 16th IEEE International Symposium on Biomedical Imaging, 236–239 (IEEE, 2019).

Bao, R., Al-Shakarji, N. M., Bunyak, F. & Palaniappan, K. DMNet: dual-stream marker guided deep network for dense cell segmentation and lineage tracking. In IEEE International Conference on Computer Vision (ICCV) Workshop on Computer Vision for Automated Medical Diagnosis, 3354–3363 (IEEE, 2021). PubMed PMC

Rahmon, G., Bunyak, F. & Palaniappan, K. Motion U-Net: multi-cue encoder–decoder network for motion segmentation. In Proceedings of the IEEE International Conference on Pattern Recognition, 8125–8132 (IEEE, 2021).

Girshick, R., Donahue, J., Darrell, T. & Malik, J. Rich feature hierarchies for accurate object detection and semantic segmentation. In Proceedings of the IEEE Conference on Computer Vision andPattern Recognition (CVPR), 580–587 (IEEE, 2014).

Wang J, et al. Deep high-resolution representation learning for visual recognition. IEEE Trans. Pattern Anal. Mach. Intell. 2021;43:3349–3364. doi: 10.1109/TPAMI.2020.2983686. PubMed DOI

Malin-Mayor C, et al. Automated reconstruction of whole-embryo lineages by learning from sparse annotations. Nat. Biotechnol. 2023;41:44–49. doi: 10.1038/s41587-022-01427-7. PubMed DOI PMC

Horn BK, Schunck BG. Determining optical flow. Artif. Intell. 1981;17:185–203. doi: 10.1016/0004-3702(81)90024-2. DOI

Dosovitskiy, A. et al. Flownet: learning optical flow with convolutional networks. In Proceedings of the IEEE International Conference on Computer Vision, 2758–2766 (IEEE, 2015).

Ranjan, A. & Black, M. J. Optical flow estimation using a spatial pyramid network. In Proceedings of theIEEE conference on Computer Vision and Pattern Recognition, 4161–4170 (IEEE, 2017).

Sun, D., Yang, X., Liu, M. Y. & Kautz, J. Pwc-net: CNNs for optical flow using pyramid, warping, and cost volume. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 8934–8943 (IEEE, 2018).

Teed, Z. & Deng, J. Raft: recurrent all-pairs field transforms for optical flow. In European Conference on Computer Vision, 402–419 (Springer, 2020).

Chan, K. C., Wang, X., Yu, K., Dong, C. & Loy, C. C. BasicVSR: the search for essential components in video super-resolution and beyond. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 4947–4956 (IEEE, 2021).

Niklaus, S., Hu, P. & Chen, J. Splatting-based synthesis for video frame interpolation. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), 713–723 (IEEE, 2023).

Osokin, A., Chessel, A., Carazo Salas, R. E. & Vaggi, F. GANs for biological image synthesis. In Proceedings of the IEEE International Conference on Computer Vision (ICCV), 2233–2241 (2017).

Magnusson, K. E. G. & Jaldén, J. Tracking of non-Brownian particles using the Viterbi algorithm. In Proceedings of the 12th IEEE International Symposium on Biomedical Imaging, 380–384 (IEEE, 2015).

Arzt M, et al. LABKIT: labeling and segmentation toolkit for big image data. Front. Comput. Sci. 2022;4:777728. doi: 10.3389/fcomp.2022.777728. DOI

Akbaş, C. E., Ulman, V., Maška, M., Jug, F. & Kozubek, M. Automatic fusion of segmentation and tracking labels. In Computer Vision – ECCV 2018 Workshops, LNCS vol. 11134, 446–454 (Springer, 2019).

Matula P, et al. Cell tracking accuracy measurement based on comparison of acyclic oriented graphs. PLoS One. 2015;10:e0144959. doi: 10.1371/journal.pone.0144959. PubMed DOI PMC

Guerrero Peña, F. A. et al. Multiclass weighted loss for instance segmentation of cluttered cells. In 25th IEEE International Conference on Image Processing (ICIP), 2451–2455 (IEEE, 2018).

Guerrero Peña, F. A., Fernandez, P. D. M., Ren, T. I. & Cunha A. A weakly supervised method for instance segmentation of biological cells. In Domain Adaptation and Representation Transfer and Medical Image Learning with Less Labels and Imperfect Data, LNCS vol. 11795, 216–224 (Springer, 2019).

Padfield D, Rittscher J, Roysam B. Coupled minimum-cost flow cell tracking for high-throughput quantitative analysis. Med. Image Anal. 2011;15:650–668. doi: 10.1016/j.media.2010.07.006. PubMed DOI

Löffler K, Scherr T, Mikut R. A graph-based cell tracking algorithm with few manually tunable parameters and automated segmentation error correction. PLoS One. 2021;16:e0249257. doi: 10.1371/journal.pone.0249257. PubMed DOI PMC

Antonelli M, et al. The medical segmentation decathlon. Nature Commun. 2022;12:4128. doi: 10.1038/s41467-022-30695-9. PubMed DOI PMC

Neven, D., Brabandere, B. D., Proesmans, M. & Van Gool, L. Instance segmentation by jointly optimizing spatial embeddings and clustering bandwidth. In Proceedings of the IEEE/CVF Conference on Computer Vision andPattern Recognition (CVPR), 8837–8845 (IEEE, 2019).

Romera E, Álvarez JM, Bergasa LM, Arroyo R. ERFNet: efficient residual factorized ConvNet for real-time semantic segmentation. IEEE Trans. Intell. Transport. Syst. 2018;19:263–272. doi: 10.1109/TITS.2017.2750080. DOI

R Core Team. R: a language and environment for statistical computinghttps://www.R-project.org/ (2017).

Topology-preserving contourwise shape fusion

Comparing Deep Learning Performance for Chronic Lymphocytic Leukaemia Cell Segmentation in Brightfield Microscopy Images