BACKGROUND: Because of its non-destructive nature, label-free imaging is an important strategy for studying biological processes. However, routine microscopic techniques like phase contrast or DIC suffer from shadow-cast artifacts making automatic segmentation challenging. The aim of this study was to compare the segmentation efficacy of published steps of segmentation work-flow (image reconstruction, foreground segmentation, cell detection (seed-point extraction) and cell (instance) segmentation) on a dataset of the same cells from multiple contrast microscopic modalities. RESULTS: We built a collection of routines aimed at image segmentation of viable adherent cells grown on the culture dish acquired by phase contrast, differential interference contrast, Hoffman modulation contrast and quantitative phase imaging, and we performed a comprehensive comparison of available segmentation methods applicable for label-free data. We demonstrated that it is crucial to perform the image reconstruction step, enabling the use of segmentation methods originally not applicable on label-free images. Further we compared foreground segmentation methods (thresholding, feature-extraction, level-set, graph-cut, learning-based), seed-point extraction methods (Laplacian of Gaussians, radial symmetry and distance transform, iterative radial voting, maximally stable extremal region and learning-based) and single cell segmentation methods. We validated suitable set of methods for each microscopy modality and published them online. CONCLUSIONS: We demonstrate that image reconstruction step allows the use of segmentation methods not originally intended for label-free imaging. In addition to the comprehensive comparison of methods, raw and reconstructed annotated data and Matlab codes are provided.

Central European Institute of Technology Brno University of Technology Purkynova 656 123 Brno CZ 612 00 Czech Republic

Department of Biomedical Engineering Faculty of Electrical Engineering and Communication Brno University of Technology Technicka 3058 10 Brno CZ 61600 Czech Republic

Department of Histology and Embryology Faculty of Medicine Masaryk University Kamenice 5 Brno CZ 62500 Czech Republic

Department of Pathological Physiology Faculty of Medicine Masaryk University Kamenice 5 Brno CZ 62500 Czech Republic

Department of Physiology Faculty of Medicine Masaryk University Kamenice 5 Brno CZ 62500 Czech Republic

International Clinical Research Center St Anne's University Hospital Pekarska 664 53 Brno CZ 65691 Czech Republic

Max Planck Institute of Molecular Cell Biology and Genetics Pfotenhauerstr 108 Dresden DE 01307 Germany

See more in PubMed

Wang Z, Millet L, Chan V, Ding H, Gillette MU, Bashir R, Popescu G. Label-free intracellular transport measured by spatial light interference microscopy. J Biomed Opt. 2011;16(2):026019–0260199. PubMed PMC

Al-Kofahi Y, Lassoued W, Lee W, Roysam B. Improved automatic detection and segmentation of cell nuclei in histopathology images. IEEE Trans Biomed Eng. 2010;57(4):841–52. PubMed

Dimopoulos S, Mayer CE, Rudolf F, Stelling J. Accurate cell segmentation in microscopy images using membrane patterns, Bioinformatics (Oxford, England) 2014;30(18):2644–51. PubMed

Chalfoun J, Majurski M, Dima A, Stuelten C, Peskin A, Brady M. FogBank: a single cell segmentation across multiple cell lines and image modalities, BMC Bioinformatics. 2014;15(1):6598. PubMed PMC

Hilsenbeck O, Schwarzfischer M, Loeffler D, Dimopoulos S, Hastreiter S, Marr C, Theis FJ, Schroeder T. Faster: a user-friendly tool for ultrafast and robust cell segmentation in large-scale microscopy. Bioinformatics. 2017;33(13):2020–28. PubMed

Wang Q, You L, West M. Celltracer: Software for automated image segmentation and line-age mapping for single-cell studies.Duke University; 2018. http://www.stat.duke.edu/~mw/MWextrapubs/Wang2008.pdf.

Stylianidou S, Brennan C, Nissen SB, Kuwada NJ, Wiggins PA. Supersegger: robust image segmentation, analysis and lineage tracking of bacterial cells. Mol Microbiol. 2016;102(4):690–700. PubMed

Bredies K, Wolinski H. An active-contour based algorithm for the automated segmentation of dense yeast populations on transmission microscopy images. Comput Vis Sci. 2011;14(7):341–52.

Versari C, Stoma S, Batmanov K, Llamosi A, Mroz F, Kaczmarek A, Deyell M, Lhoussaine C, Hersen P, Batt G. Long-term tracking of budding yeast cells in brightfield microscopy: Cellstar and the evaluation platform. J R Soc Interface. 2017;14(127):20160705. PubMed PMC

Carpenter AE, Jones TR, Lamprecht MR, Clarke C, Kang IH, Friman O, Guertin D. a., Chang JH, Lindquist R. a., Moffat J, Golland P, Sabatini DM. CellProfiler: image analysis software for identifying and quantifying cell phenotypes, Genome Biol. 2006;7(10):100. PubMed PMC

Tescan AS. Q-PHASE: Quantitative, Label-free Imaging Cytometry. https://www.tescan.com/en-us/technology/light-microscopy/q-phase. Accessed 30 Apr 2018.

Koos K, Molnár J, Kelemen L, Tamás G, Horvath P. DIC image reconstruction using an energy minimization framework to visualize optical path length distribution. Sci Rep. 2016;6:30420. PubMed PMC

Székely Gábor, Hahn Horst K., editors. Information Processing in Medical Imaging. Berlin, Heidelberg: Springer Berlin Heidelberg; 2011.

Yin Z, Kanade T, Chen M. Understanding the phase contrast optics to restore artifact-free microscopy images for segmentation. Med Image Anal. 2012;16(5):1047–62. PubMed PMC

Thirusittampalam K, Hossain MJ, Ghita O, Whelan PF. A novel framework for cellular tracking and mitosis detection in dense phase contrast microscopy images. IEEE J Biomed Health Inf. 2013;17(3):642–53. PubMed

Dewan MAA, Ahmad MO, Swamy MNS. A method for automatic segmentation of nuclei in phase-contrast images based on intensity, convexity and texture. IEEE Trans Biomed Circ Syst. 2014;8(5):716–28. PubMed

Otsu N. A threshold selection method from gray-level histograms. IEEE Trans Syst Man Cybern. 1979;9(1):62–6.

Chalfoun J, Majurski M, Peskin A, Breen C, Bajcsy P, Brady M. Empirical gradient threshold technique for automated segmentation across image modalities and cell lines. J Microsc. 2015;260(1):86–99. PubMed

Juneau P-M, Garnier A, Duchesne C. Selection and tuning of a fast and simple phase-contrast microscopy image segmentation algorithm for measuring myoblast growth kinetics in an automated manner. Microsc Microanal. 2013;19(4):855–66. PubMed

Topman G, Sharabani-Yosef O, Gefen A. A method for quick, low-cost automated confluency measurements. Microsc Microanal. 2011;17(6):915–22. PubMed

Jaccard N, Griffin LD, Keser A, Macown RJ, Super A, Veraitch FS, Szita N. Automated method for the rapid and precise estimation of adherent cell culture characteristics from phase contrast microscopy images. Biotech Bioeng. 2014;111(3):504–17. PubMed PMC

Caselles V, Kimmel R, Sapiro G. Geodesic active contours. Int J Comput Vis. 1997;22(1):61–79.

Chan TF, Vese LA. Active contours without edges. IEEE Trans Image Process. 2001;10(2):266–77. PubMed

Boykov Y, Kolmogorov V. An experimental comparison of min-cut/max- flow algorithms for energy minimization in vision. IEEE Trans Pattern Anal Mach Intell. 2004;26(9):1124–37. PubMed

Sommer C, Straehle C, Köthe U, Hamprecht FA. Ilastik: Interactive learning and segmentation toolkit. In: 2011 IEEE International Symposium on Biomedical Imaging: From Nano to Macro: 2011. p. 230–3. 10.1109/ISBI.2011.5872394.

Arganda-Carreras Ignacio, Kaynig Verena, Rueden Curtis, Eliceiri Kevin W, Schindelin Johannes, Cardona Albert, Sebastian Seung H. Trainable Weka Segmentation: a machine learning tool for microscopy pixel classification. Bioinformatics. 2017;33(15):2424–2426. PubMed

Peng H, Zhou X, Li F, Xia X, Wong STC. Integrating multi-scale blob/curvilinear detector techniques and multi-level sets for automated segmentation of stem cell images. In: 2009 IEEE International Symposium on Biomedical Imaging: From Nano to Macro. IEEE: 2009. p. 1362–5. 10.1109/ISBI.2009.5193318. http://ieeexplore.ieee.org/document/5193318/. PubMed PMC

Kong H, Akakin HC, Sarma SE. A generalized laplacian of gaussian filter for blob detection and its applications. IEEE Trans Cybern. 2013;43(6):1719–33. PubMed

Xu H, Lu C, Berendt R, Jha N, Mandal M. Automatic nuclei detection based on generalized laplacian of gaussian filters. IEEE J Biomed Health Inform. 2017;21(3):826–37. PubMed

Zhang M, Wu T, Bennett KM. Small blob identification in medical images using regional features from optimum scale. IEEE Transactions on Biomedical Engineering. 2015;62(4):1051–62. PubMed

Loy G, Zelinsky A. Fast radial symmetry for detecting points of interest. IEEE Trans Pattern Anal Mach Intell. 2003;25(8):959–73.

Bahlmann C. Proceedings of the 2012 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). CVPR ’12. Washington, DC: IEEE Computer Society; 2012. Fast radial symmetry detection under affine transformations.

Qi X, Xing F, Foran DJ, Yang L. Robust segmentation of overlapping cells in histopathology specimens using parallel seed detection and repulsive level set. IEEE Trans Biomed Eng. 2012;59(3):754–65. PubMed PMC

Matas J, Chum O, Urban M, Pajdla T. Robust wide-baseline stereo from maximally stable extremal regions. Image Vis Comput. 2004;22(10):761–7.

Arteta C, Lempitsky V, Noble JA, Zisserman A. Learning to Detect Cells Using Non-overlapping Extremal Regions. Miccai. 2012;Figure 1:1–8. PubMed

Russakovsky O, Deng J, Su H, Krause J, Satheesh S, Ma S, Huang Z, Karpathy A, Khosla A, Bernstein M, et al. Imagenet large scale visual recognition challenge. Int J Comput Vis. 2015;115(3):211–52.

Iglovikov V, Shvets A. Ternausnet: U-net with vgg11 encoder pre-trained on imagenet for image segmentation. 2018. arXiv preprint arXiv:1801.05746.

Ronneberger Olaf, Fischer Philipp, Brox Thomas. Lecture Notes in Computer Science. Cham: Springer International Publishing; 2015. U-Net: Convolutional Networks for Biomedical Image Segmentation; pp. 234–241.

Sadanandan SK, Ranefall P, Wählby C. Computer Vision – ECCV 2016 Workshops. Amsterdam: Springer International Publishing; 2016. Feature Augmented Deep Neural Networks for Segmentation of Cells. Computer Vision – ECCV 2016 Workshops.

Van Valen DA, Kudo T, Lane KM, Macklin DN, Quach NT, DeFelice MM, Maayan I, Tanouchi Y, Ashley EA, Covert MW. Deep learning automates the quantitative analysis of individual cells in live-cell imaging experiments. PLoS Comput Biol. 2016;12(11):1005177. PubMed PMC

Schmidt Uwe, Weigert Martin, Broaddus Coleman, Myers Gene. Medical Image Computing and Computer Assisted Intervention – MICCAI 2018. Cham: Springer International Publishing; 2018. Cell Detection with Star-Convex Polygons; pp. 265–273.

Xing F, Yang L. Robust nucleus/cell detection and segmentation in digital pathology and microscopy images. IEEE Rev Biomed Eng. 2016;9:234–63. PubMed PMC

Wiesmann V, Franz D, Held C, Münzenmayer C, Palmisano R, Wittenberg T. Review of free software tools for image analysis of fluorescence cell micrographs. J Microsc. 2015;257(1):39–53. PubMed

Meijering E. Cell segmentation: 50 years down the road [life sciences] IEEE Signal Process Mag. 2012;29(5):140–5.

Kasprowicz R, Suman R, O’Toole P. Characterising live cell behaviour: Traditional label-free and quantitative phase imaging approaches. Int J Biochem Cell Biol. 2017;84:89–95. PubMed

Loewke NO, Pai S, Cordeiro C, Black D, King BL, Contag CH, Chen B, Baer TM, Solgaard O. Automated cell segmentation for quantitative phase microscopy. IEEE Trans Med Imaging. 2017. PubMed PMC

Ulman V, Maška M, Magnusson KEG, Ronneberger O, Haubold C, Harder N, Matula P, Matula P, Svoboda D, Radojevic M, Smal I, Rohr K, Jaldén J, Blau HM, Dzyubachyk O, Lelieveldt B, Xiao P, Li Y, Cho S-Y, Dufour A, Olivo-Marin JC, Reyes-Aldasoro CC, Solis-Lemus JA, Bensch R, Brox T, Stegmaier J, Mikut R, Wolf S, Hamprecht FA, Esteves T, Quelhas P, Demirel Ö, Malström L, Jug F, Tomančák P, Meijering E, Muñoz-Barrutia A, Kozubek M, Ortiz-de-Solorzano C. An objective comparison of cell tracking algorithms. Nat Methods. 2017. 10.1038/NMETH.447. PubMed PMC

Kolman P, Chmelík R. Coherence-controlled holographic microscope. Opt Express. 2010;18(21):21990–2004. PubMed

Slabý T, Kolman P, Dostál Z., Antoš M., Lošťák M, Chmelík R. Off-axis setup taking full advantage of incoherent illumination in coherence-controlled holographic microscope. Opt Express. 2013;21(12):14747–62. PubMed

Davies H, Wilkins M. Interference microscopy and mass determination. Nature. 1952;169(4300):541. PubMed

Barer R. Refractometry and interferometry of living cells. J Opt Soc Am. 1957;47(6):545–56. PubMed

Koos K, Molnar J, Horvath P. DIC Microscopy Image Reconstruction Using a Novel Variational Framework. 2015. International Conference on Digital Image Computing: Techniques and Applications (DICTA). 2015. 10.1109/DICTA.2015.7371252.

Su H, Yin Z, Kanade T, Huh S. Phase contrast image restoration via dictionary representation of diffraction patterns. Med Image Comput Comput Assist Interv. 2012;15(3):615–22. PubMed

Whitaker RT. A level-set approach to 3d reconstruction from range data. Int J Comput Vis. 1998;29(3):203–31.

Breiman L. Random forests. Mach Learn. 2001;45(1):5–32.

Lindeberg T. Feature detection with automatic scale selection. Int J Comput Vis. 1998;30(2):79–116.

Ram S, Rodríguez JJ. Symmetry-based detection of nuclei in microscopy images. In: 2013 IEEE International Conference on Acoustics, Speech and Signal Processing: 2013. p. 1128–32. 10.1109/ICASSP.2013.6637826.

Parvin B, Yang Q, Han J, Chang H, Rydberg B, Barcellos-Hoff MH. Iterative voting for inference of structural saliency and characterization of subcellular events. IEEE Trans Image Process. 2007;16(3):615–23. PubMed

Lu C, Xu H, Xu J, Gilmore H, Mandal M, Madabhushi A. Multi-pass adaptive voting for nuclei detection in histopathological images. Sci Rep. 2016;6:33985. PubMed PMC

Parvati K, Rao P, Mariya Das M. Image segmentation using gray-scale morphology and marker-controlled watershed transformation. Discret Dyn Nat Soc. 2008;2008:384346.

Daněk Ondřej, Matula Pavel, Ortiz-de-Solórzano Carlos, Muñoz-Barrutia Arrate, Maška Martin, Kozubek Michal. Image Analysis. Berlin, Heidelberg: Springer Berlin Heidelberg; 2009. Segmentation of Touching Cell Nuclei Using a Two-Stage Graph Cut Model; pp. 410–419.

Zhou Y, Kuijper A, Heise B, He L. Cell Segmentation Using Level Set Method. Linz: Johannes Kepler University; 2007.

Ikonen L, Toivanen P. Shortest routes on varying height surfaces using gray-level distance transforms. Image Vis Comput. 2005;23(2):133–41.

Maška M., Ulman V, Svoboda D, Matula P, Matula P, Ederra C, Urbiola A, España T, Venkatesan S, Balak DMW, Karas P, Bolcková T., Štreitová M, Carthel C, Coraluppi S, Harder N, Rohr K, Magnusson KEG, Jaldén J, Blau HM, Dzyubachyk O, Křížek P, Hagen GM, Pastor-Escuredo D, Jimenez-Carretero D, Ledesma-Carbayo MJ, Muñoz-Barrutia A, Meijering E, Kozubek M, Ortiz-de-Solorzano C. A benchmark for comparison of cell tracking algorithms. Bioinformatics. 2014;30(11):1609–17. PubMed PMC

Self-supervised pretraining for transferable quantitative phase image cell segmentation