To stimulate progress in automating the reconstruction of neural circuits, we organized the first international challenge on 2D segmentation of electron microscopic (EM) images of the brain. Participants submitted boundary maps predicted for a test set of images, and were scored based on their agreement with a consensus of human expert annotations. The winning team had no prior experience with EM images, and employed a convolutional network. This "deep learning" approach has since become accepted as a standard for segmentation of EM images. The challenge has continued to accept submissions, and the best so far has resulted from cooperation between two teams. The challenge has probably saturated, as algorithms cannot progress beyond limits set by ambiguities inherent in 2D scoring and the size of the test dataset. Retrospective evaluation of the challenge scoring system reveals that it was not sufficiently robust to variations in the widths of neurite borders. We propose a solution to this problem, which should be useful for a future 3D segmentation challenge.

Center for Brain Science Harvard University Cambridge MA USA

Computer Science Department Rutgers University New Brunswick NJ USA

Department of Biomedical Engineering The Institute of Technology Linkoping University Linkoping Sweden

Department of Computer Science ETH Zurich Zurich Switzerland

Department of Telecommunications Faculty of Electrical Engineering and Communication Brno University of Technology Brno Czech Republic

Digital Productivity Flagship Commonwealth Scientific and Industrial Research Organisation North Ryde NSW Australia

Howard Hughes Medical Institute Janelia Research Campus Ashburn VA USA

Imaging Platform Broad Institute Cambridge MA USA

Laboratory for Optical and Computational Instrumentation University of Wisconsin Madison Madison WI USA

Princeton Neuroscience Institute and Computer Science Department Princeton University Princeton NJ USA

School of Engineering and Information Technology University of New South Wales Canberra ACT Australia

Scientific Computing and Imaging Institute University of Utah Salt Lake City UT USA

Swiss AI Lab IDSIA Universitá Della Svizzera Italiana Scuola Universitaria Professionale Della Svizzera Italiana Lugano Switzerland

UMR1318 French National Institute for Agricultural Research AgroParisTech French National Institute for Agricultural Research Centre de Versailles Grignon Institut Jean Pierre Bourgin Versailles France

Zobrazit více v PubMed

Arbeláez P., Maire M., Fowlkes C., Malik J. (2011). Contour detection and hierarchical image segmentation. IEEE Trans. Patt. Anal. Mach. Intell. 33, 898–916. 10.1109/TPAMI.2010.161 PubMed DOI

Bourne J. N., Harris K. M. (2012). Nanoscale analysis of structural synaptic plasticity. Curr. Opin. Neurobiol. 22, 372–382. 10.1016/j.conb.2011.10.019 PubMed DOI PMC

Briggman K. L., Helmstaedter M., Denk W. (2011). Wiring specificity in the direction-selectivity circuit of the retina. Nature 471, 183–188. 10.1038/nature09818 PubMed DOI

Bullinger A. C., Neyer A.-K., Rass M., Moeslein K. M. (2010). Community-based innovation contests: Where competition meets cooperation. Creat. Innov. Manage. 19, 290–303. 10.1111/j.1467-8691.2010.00565.x DOI

Bumbarger D. J., Riebesell M., Rödelsperger C., Sommer R. J. (2013). System-wide rewiring underlies behavioral differences in predatory and bacterial-feeding nematodes. Cell 152, 109–119. 10.1016/j.cell.2012.12.013 PubMed DOI

Cardona A., Saalfeld S., Preibisch S., Schmid B., Cheng A., Pulokas, et al. . (2010). An integrated micro-and macroarchitectural analysis of the drosophila brain by computer-assisted serial section electron microscopy.PLoS Biol. 8:e1000502. 10.1371/journal.pbio.1000502 PubMed DOI PMC

Cardona A., Saalfeld S., Schindelin J., Arganda-Carreras I., Preibisch S., Longair M., et al. . (2012). Trakem2 software for neural circuit reconstruction. PLoS ONE 7:e38011. 10.1371/journal.pone.0038011 PubMed DOI PMC

Chenouard N., Smal I., de Chaumont F., Maška M., Sbalzarini I. F., Gong Y., et al. . (2014). Objective comparison of particle tracking methods. Nat. Methods 11, 281–289. 10.1038/nmeth.2808 PubMed DOI PMC

Chklovskii D., Vitaladevuni S., Scheffer L. (2010). Semi-automated reconstruction of neural circuits using electron microscopy. Curr. Opin. Neurobiol. 20, 667–675. 10.1016/j.conb.2010.08.002 PubMed DOI

Ciresan D. C., Giusti A., Gambardella L. M., Schmidhuber J. (2012). Deep neural networks segment neuronal membranes in electron microscopy images, in Annual Conference on Neural Information Processing Systems (NIPS) (Lake Tahoe, NV: ), 2852–2860.

Funke J., Andres B., Hamprecht F., Cardona A., Cook M. (2012). Efficient automatic 3D-reconstruction of branching neurons from EM data. Comput. Vis. Patt. Recogn. 1004–1011 10.1109/cvpr.2012.6247777 DOI

Helmstaedter M., Briggman K. L., Turaga S. C., Jain V., Seung H. S., Denk W. (2013). Connectomic reconstruction of the inner plexiform layer in the mouse retina. Nature 500, 168–174. 10.1038/nature12346 PubMed DOI

Jain V., Bollmann B., Richardson M., Berger D. R., Helmstaedter M. N., Briggman K. L., et al. (2010a). Boundary learning by optimization with topological constraints, in 2010 IEEE Conference On Computer Vision and Pattern Recognition (CVPR) (San Francisco, CA: IEEE Computer Society; ), 2488–2495.

Jain V., Murray J., Roth F., Turaga S., Zhigulin V., Briggman K., et al. (2007). Supervised learning of image restoration with convolutional networks, in IEEE 11th International Conference on Computer Vision, ICCV (Rio de Janeiro: ), 1–8.

Jain V., Seung H. S., Turaga S. C. (2010b). Machines that learn to segment images: a crucial technology for connectomics. Curr. Opin. Neurobiol. 20, 653–666. 10.1016/j.conb.2010.07.004 PubMed DOI PMC

Jarrell T. A., Wang Y., Bloniarz A. E., Brittin C. A., Xu M., Thomson J. N., et al. . (2012). The connectome of a decision-making neural network. Science 337, 437–444. 10.1126/science.1221762 PubMed DOI

Kasthuri N., Hayworth K. J., Berger D. R., Schalek R. L., Conchello J. A., Knowles-Barley S., et al. . (2015). Saturated reconstruction of a volume of neocortex. Cell 162, 648–661. 10.1016/j.cell.2015.06.054 PubMed DOI

Kaynig V., Vazquez-Reina A., Knowles-Barley S., Roberts M., Jones T. R., Kasthuri N., et al. . (2015). Large-scale automatic reconstruction of neuronal processes from electron microscopy images. Med. Image Anal. 22, 77–88. 10.1016/j.media.2015.02.001 PubMed DOI PMC

Kim J. S., Greene M. J., Zlateski A., Lee K., Richardson M., Turaga S. C., et al. . (2014). Space-time wiring specificity supports direction selectivity in the retina. Nature 509, 331–336. 10.1038/nature13240 PubMed DOI PMC

Krizhevsky A., Sutskever I., Hinton G. E. (2013). Imagenet classification with deep convolutional neural networks. Adv. Neural Inf. Process. Syst. 25, 1097–1105.

Kroeger T., Mikula S., Denk W., Koethe U., Hamprecht F. A. (2013). Learning to segment neurons with non-local quality measures. Med. Image Comput. Comput. Assist. Interv. 16(Pt 2), 419–427. 10.1007/978-3-642-40763-5_52 PubMed DOI

Lakhani K. R., Boudreau K. J., Loh P.-R., Backstrom L., Baldwin C., Lonstein E., et al. . (2013). Prize-based contests can provide solutions to computational biology problems. Nat. Biotechnol. 31, 108–111. 10.1038/nbt.2495 PubMed DOI PMC

LeCun Y., Boser B., Denker J. S., Henderson D., Howard R. E., Hubbard W., et al. (1989). Backpropagation applied to handwritten zip code recognition. Neural Comput. 1, 541–551.

LeCun Y., Huang F. J., Bottou L. (2004). Learning methods for generic object recognition with invariance to pose and lighting, in IEEE Conference on Computer Vision and Pattern Recognition, CVPR (Washington, DC: IEEE Computer Society; ), 97–104.

Martin D., Fowlkes C., Malik J. (2004). Learning to detect natural image boundaries using local brightness, color, and texture cues. IEEE Trans. Patt. Anal. Mach. Intell. 26, 530–549. 10.1109/TPAMI.2004.1273918 PubMed DOI

Meilǎ M. (2005). Comparing clusterings: an axiomatic view, in Proceedings of the 22nd International Conference on Machine Learning, ICML '05 (New York, NY: ACM; ), 577–584.

Mishchenko Y., Hu T., Spacek J., Mendenhall J., Harris K. M., Chklovskii D. B. (2010). Ultrastructural analysis of hippocampal neuropil from the connectomics perspective. Neuron 67, 1009–1020. 10.1016/j.neuron.2010.08.014 PubMed DOI PMC

Nunez-Iglesias J., Kennedy R., Parag T., Shi J., Chklovskii D. B. (2013). Machine learning of hierarchical clustering to segment 2D and 3D images. PLoS ONE 8:e71715. 10.1371/journal.pone.0071715 PubMed DOI PMC

Rand W. M. (1971). Objective criteria for the evaluation of clustering methods. J. Am. Stat. Assoc. 66, 846–850.

Russakovsky O., Deng J., Su H., Krause J., Satheesh S., Ma S., et al. . (2014). Imagenet large scale visual recognition challenge. Int. J. Comput. Vis. 1–42. PubMed

Schindelin J., Arganda-Carreras I., Frise E., Kaynig V., Longair M., Pietzsch T., et al. . (2012). Fiji: an open-source platform for biological-image analysis. Nat. Methods 9, 676–682. 10.1038/nmeth.2019 PubMed DOI PMC

Suloway C., Pulokas J., Fellmann D., Cheng A., Guerra F., Quispe J., et al. . (2005). Automated molecular microscopy: the new Leginon system. J. Struct. Biol. 151, 41–60. 10.1016/j.jsb.2005.03.010 PubMed DOI

Takemura S.-Y., Bharioke A., Lu Z., Nern A., Vitaladevuni S., Rivlin P. K., et al. . (2013). A visual motion detection circuit suggested by Drosophila connectomics. Nature 500, 175–181. 10.1038/nature12450 PubMed DOI PMC

Tapia J. C., Wylie J. D., Kasthuri N., Hayworth K. J., Schalek R., Berger D. R., et al. . (2012). Pervasive synaptic branch removal in the mammalian neuromuscular system at birth. Neuron 74, 816–829. 10.1016/j.neuron.2012.04.017 PubMed DOI

Turaga S., Briggman K., Helmstaedter M., Denk W., Seung H. S. (2009). Maximin affinity learning of image segmentation. Adv. Neural Info. Proc. Syst. 22, 1865–1873.

Unnikrishnan R., Pantofaru C., Hebert M. (2007). Toward objective evaluation of image segmentation algorithms. IEEE Trans. Patt. Anal. Mach. Intell. 29, 929–944. 10.1109/TPAMI.2007.1046 PubMed DOI

White J. G., Southgate E., Thomson J. N., Brenner S. (1986). The structure of the nervous system of the nematode Caenorhabditis elegans. Philos. Trans. R. Soc. Lond. B Biol. Sci. 314, 1–340. PubMed