Particle tracking is of key importance for quantitative analysis of intracellular dynamic processes from time-lapse microscopy image data. Because manually detecting and following large numbers of individual particles is not feasible, automated computational methods have been developed for these tasks by many groups. Aiming to perform an objective comparison of methods, we gathered the community and organized an open competition in which participating teams applied their own methods independently to a commonly defined data set including diverse scenarios. Performance was assessed using commonly defined measures. Although no single method performed best across all scenarios, the results revealed clear differences between the various approaches, leading to notable practical conclusions for users and developers.

] Center for Applied Medical Research University of Navarra Pamplona Spain [2] Centre for Biomedical Image Analysis Masaryk University Brno Czech Republic [3]

] Department of Bioinformatics and Functional Genomics Institute of Pharmacy and Molecular Biotechnology University of Heidelberg Heidelberg Germany [2] Division of Theoretical Bioinformatics German Cancer Research Center Heidelberg Germany

] Department of Medical Informatics Erasmus University Medical Center Rotterdam The Netherlands [2] Department of Radiology Erasmus University Medical Center Rotterdam The Netherlands [3]

] Institut Pasteur Unité d'Analyse d'Images Quantitative Centre National de la Recherche Scientifique Unité de Recherche Associée 2582 Paris France [2]

] Institut Pasteur Unité d'Analyse d'Images Quantitative Centre National de la Recherche Scientifique Unité de Recherche Associée 2582 Paris France [2] Biomedical Imaging Group École Polytechnique Fédérale de Lausanne Lausanne Switzerland [3] New York University Neuroscience Institute New York University Medical Center New York New York USA [4]

] Max Planck Institute of Molecular Cell Biology and Genetics Dresden Germany [2] Belozersky Institute of Physico Chemical Biology Moscow State University Moscow Russia

Baxter Laboratory for Stem Cell Biology Department of Microbiology and Immunology Stanford University School of Medicine Stanford California USA

Cell and Tissue Imaging Facility Institut Curie Paris France

Center for Applied Medical Research University of Navarra Pamplona Spain

Compunetix Inc Monroeville Pennsylvania USA

Department of Biomedical Engineering Chung Yuan Christian University Chung Li City Taiwan China

Department of Biomedical Engineering Zhejiang University Hangzhou China

Department of Cell Biology Yale University New Haven Connecticut USA

Department of Electrical and Computer Engineering Drexel University Philadelphia Pennsylvania USA

Department of Electrical Engineering Yale University New Haven Connecticut USA

Department of Signal Processing ACCESS Linnaeus Centre KTH Royal Institute of Technology Stockholm Sweden

Inria Rennes Bretagne Atlantique Rennes France

Molecular Biotechnology Group Institute of Biology Leiden University Leiden The Netherlands

MOSAIC Group Max Planck Institute of Molecular Cell Biology and Genetics Dresden Germany

Plateforme d'Imagerie Dynamique Imagopole Institut Pasteur Paris France

Comment In

Nat Methods. 2014 Mar;11(3):247-8. doi: 10.1038/nmeth.2851. PubMed

See more in PubMed

Saxton MJ, Jacobson K. Single-particle tracking: applications to membrane dynamics. Annu. Rev. Biophys. Biomol. Struct. 1997;26:373–399. doi: 10.1146/annurev.biophys.26.1.373. PubMed DOI

Akhmanova A, Steinmetz MO. Tracking the ends: a dynamic protein network controls the fate of microtubule tips. Nat. Rev. Mol. Cell Biol. 2008;9:309–322. doi: 10.1038/nrm2369. PubMed DOI

Berginski ME, Vitriol EA, Hahn KM, Gomez SM. High-resolution quantification of focal adhesion spatiotemporal dynamics in living cells. PLoS ONE. 2011;6:e22025. doi: 10.1371/journal.pone.0022025. PubMed DOI PMC

Brandenburg B, Zhuang X. Virus trafficking–learning from single-virus tracking. Nat. Rev. Microbiol. 2007;5:197–208. doi: 10.1038/nrmicro1615. PubMed DOI PMC

Jandt U, Zeng A-P. Modeling of intracellular transport and compartmentation. Adv. Biochem. Eng. Biotechnol. 2012;127:221–249. PubMed

Sinha B, et al. Dynamic organization of chromatin assembly and transcription factories in living cells. Methods Cell Biol. 2010;98:57–78. doi: 10.1016/S0091-679X(10)98003-5. PubMed DOI

Agarwal S, et al. ATP-dependent and independent functions of Rad54 in genome maintenance. J. Cell Biol. 2011;192:735–750. doi: 10.1083/jcb.201011025. PubMed DOI PMC

Stephens DJ, Allan VJ. Light microscopy techniques for live cell imaging. Science. 2003;300:82–86. doi: 10.1126/science.1082160. PubMed DOI

Ji N, Shroff H, Zhong H, Betzig E. Advances in the speed and resolution of light microscopy. Curr. Opin. Neurobiol. 2008;18:605–616. doi: 10.1016/j.conb.2009.03.009. PubMed DOI

Shaner NC, Steinbach PA, Tsien RY. A guide to choosing fluorescent proteins. Nat. Methods. 2005;2:905–909. doi: 10.1038/nmeth819. PubMed DOI

Giepmans BNG, Adams SR, Ellisman MH, Tsien RY. The fluorescent toolbox for assessing protein location and function. Science. 2006;312:217–224. doi: 10.1126/science.1124618. PubMed DOI

Saxton MJ. Single-particle tracking: connecting the dots. Nat. Methods. 2008;5:671–672. doi: 10.1038/nmeth0808-671. PubMed DOI

Genovesio A, et al. Multiple particle tracking in 3-D+t microscopy: method and application to the tracking of endocytosed quantum dots. IEEE Trans. Image Process. 2006;15:1062–1070. doi: 10.1109/TIP.2006.872323. PubMed DOI

Smal I, Draegestein K, Galjart N, Niessen W, Meijering E. Particle filtering for multiple object tracking in dynamic fluorescence microscopy images: application to microtubule growth analysis. IEEE Trans. Med. Imaging. 2008;27:789–804. doi: 10.1109/TMI.2008.916964. PubMed DOI

Kalaidzidis Y. Multiple objects tracking in fluorescence microscopy. J. Math. Biol. 2009;58:57–80. doi: 10.1007/s00285-008-0180-4. PubMed DOI PMC

Meijering E, Dzyubachyk O, Smal I. Methods for cell and particle tracking. Methods Enzymol. 2012;504:183–200. doi: 10.1016/B978-0-12-391857-4.00009-4. PubMed DOI

Meijering E, Smal I, Danuser G. Tracking in molecular bioimaging. IEEE Signal Process. Mag. 2006;23:46–53. doi: 10.1109/MSP.2006.1628877. DOI

Kalaidzidis Y. Intracellular objects tracking. Eur. J. Cell Biol. 2007;86:569–578. doi: 10.1016/j.ejcb.2007.05.005. PubMed DOI

Dorn JF, Danuser G, Yang G. Computational processing and analysis of dynamic fluorescence image data. Methods Cell Biol. 2008;85:497–538. doi: 10.1016/S0091-679X(08)85022-4. PubMed DOI

Meijering E, Dzyubachyk O, Smal I, van Cappellen WA. Tracking in cell and developmental biology. Semin. Cell Dev. Biol. 2009;20:894–902. doi: 10.1016/j.semcdb.2009.07.004. PubMed DOI

Jaqaman K, Danuser G. Computational image analysis of cellular dynamics: a case study based on particle tracking. Cold Spring Harb. Protoc. 2009;2009:pdb.top65. doi: 10.1101/pdb.top65. PubMed DOI PMC

Rohr K, et al. Tracking and quantitative analysis of dynamic movements of cells and particles. Cold Spring Harb. Protoc. 2010;2010:pdb.top80. doi: 10.1101/pdb.top80. PubMed DOI

Blackman, S. & Popoli, R. Design and Analysis of Modern Tracking Systems (Artech House, Norwood, Massachusetts, USA, 1999).

Sonka, M., Hlavac, V. & Boyle, R. Image Processing, Analysis, and Machine Vision 3rd edn. (Cengage Learning, Florence, Kentucky, USA, 2007).

Cheezum MK, Walker WF, Guilford WH. Quantitative comparison of algorithms for tracking single fluorescent particles. Biophys. J. 2001;81:2378–2388. doi: 10.1016/S0006-3495(01)75884-5. PubMed DOI PMC

Carter BC, Shubeita GT, Gross SP. Tracking single particles: a user-friendly quantitative evaluation. Phys. Biol. 2005;2:60–72. doi: 10.1088/1478-3967/2/1/008. PubMed DOI

Smal I, Loog M, Niessen W, Meijering E. Quantitative comparison of spot detection methods in fluorescence microscopy. IEEE Trans. Med. Imaging. 2010;29:282–301. doi: 10.1109/TMI.2009.2025127. PubMed DOI

Ruusuvuori P, et al. Evaluation of methods for detection of fluorescence labeled subcellular objects in microscope images. BMC Bioinformatics. 2010;11:248. doi: 10.1186/1471-2105-11-248. PubMed DOI PMC

Godinez WJ, et al. Deterministic and probabilistic approaches for tracking virus particles in time-lapse fluorescence microscopy image sequences. Med. Image Anal. 2009;13:325–342. doi: 10.1016/j.media.2008.12.004. PubMed DOI

Gillette TA, Brown KM, Svoboda K, Liu Y, Ascoli GA. DIADEMchallenge.org: a compendium of resources fostering the continuous development of automated neuronal reconstruction. Neuroinformatics. 2011;9:303–304. doi: 10.1007/s12021-011-9104-3. DOI

Anonymous. Going for algorithm gold. Nat. Methods5, 659 (2008). PubMed

Sbalzarini IF, Koumoutsakos P. Feature point tracking and trajectory analysis for video imaging in cell biology. J. Struct. Biol. 2005;151:182–195. doi: 10.1016/j.jsb.2005.06.002. PubMed DOI

Coraluppi S, Carthel C. Recursive track fusion for multi-sensor surveillance. Inf. Fusion. 2004;5:23–33. doi: 10.1016/j.inffus.2003.03.003. DOI

Coraluppi S, Carthel C. Multi-stage multiple-hypothesis tracking. J. Adv. Inf. Fusion. 2011;6:57–67.

Olivo-Marin J-C. Extraction of spots in biological images using multiscale products. Pattern Recognit. 2002;35:1989–1996. doi: 10.1016/S0031-3203(01)00127-3. DOI

Chenouard, N., Bloch, I. & Olivo-Marin, J.-C. Multiple hypothesis tracking in cluttered condition. in Proc. Int. Conf. Image Proc. 3621–3624 (IEEE, 2009).

Chenouard, N., Bloch, I. & Olivo-Marin, J.-C. Multiple hypothesis tracking in microscopy images. in Proc. IEEE Int. Symp. Biomed. Imaging 1346–1349 (IEEE, 2009).

Winter MR, Fang C, Banker G, Roysam B, Cohen AR. Axonal transport analysis using Multitemporal Association Tracking. Int. J. Comput. Biol. Drug Des. 2012;5:35–48. doi: 10.1504/IJCBDD.2012.045950. PubMed DOI PMC

Winter M, et al. Vertebrate neural stem cell segmentation, tracking and lineaging with validation and editing. Nat. Protoc. 2011;6:1942–1952. doi: 10.1038/nprot.2011.422. PubMed DOI PMC

Godinez, W.J., Lampe, M., Eils, R., Müller, B. & Rohr, K. Tracking multiple particles in fluorescence microscopy images via probabilistic data association. in Proc. IEEE Int. Symp. Biomed. Imaging 1925–1928 (IEEE, 2011). PubMed

Rink J, Ghigo E, Kalaidzidis Y, Zerial M. Rab conversion as a mechanism of progression from early to late endosomes. Cell. 2005;122:735–749. doi: 10.1016/j.cell.2005.06.043. PubMed DOI

Liang L, Shen H, De Camilli P, Duncan JS. Tracking clathrin coated pits with a multiple hypothesis based method. Med. Image Comput. Comput. Assist. Interv. 2010;6362:315–322. PubMed PMC

Yin Z, Kanade T, Chen M. Understanding the phase contrast optics to restore artifact-free microscopy images for segmentation. Med. Image Anal. 2012;16:1047–1062. doi: 10.1016/j.media.2011.12.006. PubMed DOI PMC

Magnusson, K.E.G. & Jaldén, J. A batch algorithm using iterative application of the Viterbi algorithm to track cells and construct cell lineages. in Proc. IEEE Int. Symp. Biomed. Imaging 382–385 (IEEE, 2012).

Husain M, Boudier T, Paul-Gilloteaux P, Casuso I, Scheuring S. Software for drift compensation, particle tracking and particle analysis of high-speed atomic force microscopy image series. J. Mol. Recognit. 2012;25:292–298. doi: 10.1002/jmr.2187. PubMed DOI

Casuso I, et al. Characterization of the motion of membrane proteins using high-speed atomic force microscopy. Nat. Nanotechnol. 2012;7:525–529. doi: 10.1038/nnano.2012.109. PubMed DOI

Rao AR, Schunck BG. Computing oriented texture fields. CVGIP: Graph. Models Image Process. 1991;53:157–185.

Hager, G.D., Dewan, M. & Stewart, C.V. Multiple kernel tracking with SSD. in Proc. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit. 790–797 (IEEE, 2004).

Rousseeuw, P.J. & Leroy, A.M. Robust Regression and Outlier Detection (Wiley, Hoboken, New Jersey, USA, 2003).

Thompson RE, Larson DR, Webb WW. Precise nanometer localization analysis for individual fluorescent probes. Biophys. J. 2002;82:2775–2783. doi: 10.1016/S0006-3495(02)75618-X. PubMed DOI PMC

Shafique K, Shah M. A noniterative greedy algorithm for multiframe point correspondence. IEEE Trans. Pattern Anal. Mach. Intell. 2005;27:51–65. doi: 10.1109/TPAMI.2005.1. PubMed DOI

Jaqaman K, et al. Robust single-particle tracking in live-cell time-lapse sequences. Nat. Methods. 2008;5:695–702. doi: 10.1038/nmeth.1237. PubMed DOI PMC

Lowe DG. Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 2004;60:91–110. doi: 10.1023/B:VISI.0000029664.99615.94. DOI

Crocker JC, Grier DG. Methods of digital video microscopy for colloidal studies. J. Colloid Interface Sci. 1996;179:298–310. doi: 10.1006/jcis.1996.0217. DOI

Celler K, van Wezel GP, Willemse J. Single particle tracking of dynamically localizing TatA complexes in Streptomyces coelicolor. Biochem. Biophys. Res. Commun. 2013;438:38–42. doi: 10.1016/j.bbrc.2013.07.016. PubMed DOI

Ku T-C, et al. An automated tracking system to measure the dynamic properties of vesicles in living cells. Microsc. Res. Tech. 2007;70:119–134. doi: 10.1002/jemt.20392. PubMed DOI

Ku T-C, Kao L-S, Lin C-C, Tsai Y-S. Morphological filter improve the efficiency of automated tracking of secretory vesicles with various dynamic properties. Microsc. Res. Tech. 2009;72:639–649. doi: 10.1002/jemt.20711. PubMed DOI

Huth J, et al. Significantly improved precision of cell migration analysis in time-lapse video microscopy through use of a fully automated tracking system. BMC Cell Biol. 2010;11:24. doi: 10.1186/1471-2121-11-24. PubMed DOI PMC

Munkres J. Algorithms for the assignment and transportation problems. J. Soc. Ind. Appl. Math. 1957;5:32–38. doi: 10.1137/0105003. DOI

Saxton MJ. Wanted: a positive control for anomalous subdiffusion. Biophys. J. 2012;103:2411–2422. doi: 10.1016/j.bpj.2012.10.038. PubMed DOI PMC

de Chaumont F, et al. Icy: an open bioimage informatics platform for extended reproducible research. Nat. Methods. 2012;9:690–696. doi: 10.1038/nmeth.2075. PubMed DOI

Schindelin J, et al. Fiji: an open-source platform for biological-image analysis. Nat. Methods. 2012;9:676–682. doi: 10.1038/nmeth.2019. PubMed DOI PMC

Optimized molecule detection in localization microscopy with selected false positive probability

Metrics reloaded: recommendations for image analysis validation

The ImageJ ecosystem: Open-source software for image visualization, processing, and analysis

BIAS: Transparent reporting of biomedical image analysis challenges

BIAFLOWS: A Collaborative Framework to Reproducibly Deploy and Benchmark Bioimage Analysis Workflows

CytoPacq: a web-interface for simulating multi-dimensional cell imaging

Why rankings of biomedical image analysis competitions should be interpreted with care

Mason-Pfizer Monkey Virus Envelope Glycoprotein Cycling and Its Vesicular Co-Transport with Immature Particles

An objective comparison of cell-tracking algorithms

Cell Tracking Accuracy Measurement Based on Comparison of Acyclic Oriented Graphs

Crowdsourcing the creation of image segmentation algorithms for connectomics

Localized movement and morphology of UBF1-positive nucleolar regions are changed by γ-irradiation in G2 phase of the cell cycle