It is now possible to generate large volumes of high-quality images of biomolecules at near-atomic resolution and in near-native states using cryogenic electron microscopy/electron tomography (Cryo-EM/ET). However, the precise annotation of structures like filaments and membranes remains a major barrier towards applying these methods in high-throughput. To address this, we present TARDIS (Transformer-based Rapid Dimensionless Instance Segmentation), a machine-learning framework for fast and accurate annotation of micrographs and tomograms. TARDIS combines deep learning for semantic segmentation with a novel geometric model for precise instance segmentation of various macromolecules. We develop pre-trained models within TARDIS for segmenting microtubules and membranes, demonstrating high accuracy across multiple modalities and resolutions, enabling segmentation of over 13,000 tomograms from the CZI Cryo-Electron Tomography data portal. As a modular framework, TARDIS can be extended to new structures and imaging modalities with minimal modification. TARDIS is open-source and freely available at https://github.com/SMLC-NYSBC/TARDIS, and accelerates analysis of high-resolution biomolecular structural imaging data.

Center for Computational Biology Flatiron Institute New York United State

Center for Membrane and Cell Physiology University of Virginia School of Medicine Charlottesville United States

Centre for Bioimaging Sciences Department of Biological Sciences National University of Singapore Singapore Singapore

Core Facility Cellular Imaging Faculty of Medicine Carl Gustav Carus Technische Universität Dresden Dresden Germany

Department of Anesthesiology Columbia University Irving Medical Center New York United States

Department of Cell Biology University of Virginia School of Medicine Charlottesville United States

Department of Chemistry and Biochemistry City College of New York United States

Department of Molecular Physiology and Biological Physics University of Virginia School of Medicine Charlottesville United States

Department of Physiology and Cellular Biophysics Columbia University Irving Medical Center New York United States

Experimental Center Faculty of Medicine Carl Gustav Carus Technische Universität Dresden Dresden Germany

Institute of Biotechnology of the Czech Academy of Sciences BIOCEV Vestec Czech Republic

Programme in Emerging Infectious Diseases Duke National University of Singapore Medical School Singapore Singapore

Simons Electron Microscopy Center New York Structural Biology Center New York United States

Simons Machine Learning Center New York Structural Biology Center New York United States

Structural Biology Initiative CUNY Advanced Science Research Center New York United States

Zobrazit více v PubMed

Steere R. L. Electron microscopy of structural detail in frozen biological specimens. J. Biophys. Biochem. Cytol. 3, 45–60 (1957). PubMed PMC

Milne J. L. S. et al. Cryo-electron microscopy--a primer for the non-microscopist. FEBS J. 280, 28–45 (2013). PubMed PMC

Mastronarde D. N. Dual-Axis Tomography: An Approach with Alignment Methods ThatPreserve Resolution. Journal of Structural Biology 120, 343–352 (December 1997). PubMed

McEwen B. F. & Marko M. Chapter 5 Three-Dimensional Transmission Electron Microscopy and Its Application to Mitosis Research. in 81–111 (1998). doi:10.1016/S0091-679X(08)61976-7. PubMed DOI

Peddie C. J. et al. Volume electron microscopy. Nat. Rev. Methods Primers 2, 51 (2022). PubMed PMC

Doerr A. Cryo-electron tomography. Nat. Methods 14, 34–34 (2017).

Nogales E. & Mahamid J. Bridging structural and cell biology with cryo-electron microscopy. Nature 628, 47–56 (2024). PubMed PMC

Tollervey F., Rios M. U., Zagoriy E., Woodruff J. B. & Mahamid J. Native molecular architectures of centrosomes in C. elegans embryos. bioRxivorg (2024) doi:10.1101/2024.04.03.587742. PubMed DOI PMC

Berger C. et al. Cryo-electron tomography on focused ion beam lamellae transforms structural cell biology. Nat. Methods 20, 499–511 (2023). PubMed

Kiewisz R. et al. Three-dimensional structure of kinetochore-fibers in human mitotic spindles. Elife 11, (July 2022). PubMed PMC

Redemann S. et al. C. elegans chromosomes connect to centrosomes by anchoring into the spindle network. Nat. Commun. 8, 15288 (August 2017). PubMed PMC

Yu C.-H. et al. Central-spindle microtubules are strongly coupled to chromosomes during both anaphase A and anaphase B. Molecular Biology of the Cell 30, 2503–2514 (September 2019). PubMed PMC

Xu C., Han W. & Cong Y. Cryo-EM and cryo-ET of the spike, virion, and antibody neutralization of SARS-CoV-2 and VOCs. Curr. Opin. Struct. Biol. 82, 102664 (2023). PubMed

van Hoorn C. & Carter A. P. A cryo-ET study of ciliary rootlet organization. (2024) doi:10.7554/elife.91642. PubMed DOI PMC

Creekmore B. C., Kixmoeller K., Black B. E., Lee E. B. & Chang Y.-W. Ultrastructure of human brain tissue vitrified from autopsy revealed by cryo-ET with cryo-plasma FIB milling. Nat. Commun. 15, 2660 (2024). PubMed PMC

Guaita M., Watters S. C. & Loerch S. Recent advances and current trends in cryo-electron microscopy. Curr. Opin. Struct. Biol. 77, 102484 (2022). PubMed PMC

Iudin A. et al. EMPIAR: The electron microscopy public image archive. Nucleic Acids Res. 51, D1503–D1511 (2023). PubMed PMC

The Chan Zuckerberg Institute. Cryo-ET Data Portal. Preprint at (2023).

Ermel U. et al. A data portal for providing standardized annotations for cryo-electron tomography. Nat. Methods (2024) doi:10.1038/s41592-024-02477-2. PubMed DOI

Eisenstein F. et al. Parallel cryo electron tomography on in situ lamellae. Nat. Methods 20, 131–138 (2023). PubMed

Kiewisz R., Baum D., Müller-Reichert T. & Fabig G. Serial-section Electron Tomography and Quantitative Analysis of Microtubule Organization in 3D-reconstructed Mitotic Spindles. Bio Protoc. 13, (2023). PubMed PMC

wwPDB Consortium. EMDB-the Electron Microscopy Data Bank. Nucleic Acids Res. 52, D456–D465 (2024). PubMed PMC

Gudimchuk N. B. et al. Mechanisms of microtubule dynamics and force generation examined with computational modeling and electron cryotomography. Nat. Commun. 11, 1–15 (July 2020). PubMed PMC

McLaren M. et al. CryoEM reveals that ribosomes in microsporidian spores are locked in a dimeric hibernating state. Nat. Microbiol. 8, 1834–1845 (2023). PubMed PMC

Franken L. E. et al. A general mechanism of ribosome dimerization revealed by single-particle cryo-electron microscopy. Nat. Commun. 8, (2017). PubMed PMC

Bepler T. et al. Antiviral activity of the host defense peptide piscidin 1: investigating a membrane-mediated mode of action. Front. Chem. 12, 1379192 (2024). PubMed PMC

Fabig G. et al. Male meiotic spindle features that efficiently segregate paired and lagging chromosomes. eLife 9, (March 2020). PubMed PMC

Redemann S. & Nazockdast E. Integrated 3D Tomography and Computational Modeling to Study Forces in Metaphase Spindles. FASEB J. 34, 1–1 (April 2020).

Mendonça L. et al. Correlative multi-scale cryo-imaging unveils SARS-CoV-2 assembly and egress. Nat. Commun. 12, 4629 (2021). PubMed PMC

Rast A. et al. Biogenic regions of cyanobacterial thylakoids form contact sites with the plasma membrane. Nat. Plants 5, 436–446 (2019). PubMed

Hylton R. K. & Swulius M. T. Challenges and triumphs in cryo-electron tomography. iScience 24, 102959 (2021). PubMed PMC

Zimyanin V. et al. Using 3D large scale tomography to study force generation in the mitotic spindle. Microsc. Microanal. 30, (2024).

Heinrich L. et al. Whole-cell organelle segmentation in volume electron microscopy. Nature 599, 141–146 (2021). PubMed

Schindelin J. et al. Fiji: an open-source platform for biological-image analysis. NatureMethods 9, 676–682 (July 2012). PubMed PMC

Contributors N. napari: a multi-dimensional image viewer for python. Zenodo Preprint at (2019).

Kremer J. R., Mastronarde D. N. & McIntosh J. R. Computer Visualization of Three-Dimensional Image Data Using IMOD. Journal of Structural Biology 116, 71–76 (January 1996). PubMed

Martinez-Sanchez A., Garcia I., Asano S., Lucic V. & Fernandez J.-J. Robust membrane detection based on tensor voting for electron tomography. J. Struct. Biol. 186, 49–61 (2014). PubMed

Lorenz L. et al. MemBrain v2: an end-to-end tool for the analysis of membranes in cryo-electron tomography. bioRxivorg (2023).

Stalling D., Westerhoff M. & Hege H.-C. Amira: a Highly Interactive System for VisualData Analysis. in (eds. Hansen C. D. & Johnson C. R.) 749–767 (Elsevier, 2005).

Inc. C. T. C. Dragonfly. Preprint at (2022).

Kiewisz R., Fabig G., Müller-Reichert T. & Bepler T. Automated Segmentation of 3D Cytoskeletal Filaments from Electron Micrographs with TARDIS. Microsc. Microanal. 29, 970–972 (July 2023).

Kiewisz R. & Bepler T. Membrane and microtubule rapid instance segmentation with dimensionless instance segmentation by learning graph representations of point clouds. NeurIPS Machine Learning in Structural Biology Workshop Preprint at (2022).

Lomonossoff G. P. & Wege C. TMV particles: The journey from fundamental studies to bionanotechnology applications. Adv. Virus Res. 102, 149–176 (2018). PubMed PMC

Franklin R. E. Structure of tobacco mosaic virus: Location of the ribonucleic acid in the tobacco mosaic virus particle. Nature 177, 928–930 (1956).

NeurIPS Preparation Of Labeled Cryo-ET Datasets For Training And Evaluation Of Machine Learning Models. https://neurips.cc/virtual/2023/77456.

Banks E. J. et al. Asymmetric peptidoglycan editing generates cell curvature in Bdellovibrio predatory bacteria. Nat. Commun. 13, 1509 (2022). PubMed PMC

Foster H. E., Ventura Santos C. & Carter A. P. A cryo-ET survey of microtubules and intracellular compartments in mammalian axons. J. Cell Biol. 221, (2022). PubMed PMC

Ooi J. S. G. et al. Tomography studies of the dengue fusion process. Acta Crystallogr. A Found. Adv. 79, C566–C566 (August 2023).

Redemann S. et al. The Segmentation of Microtubules in Electron Tomograms Using Amira. in (ed. Sharp D. J.) vol. 1136 261–278 (Springer Science+ Business Media, 2014). PubMed

Nogales E. & Downing K. H. Tubulin and Microtubule Structures. in 211–225 (Humana Press, January 2009). doi:10.1007/978-1-59745-336-3_9. DOI

Jiang T. & Cheng J. Target recognition based on CNN with LeakyReLU and PReLU activation functions. in 2019 International Conference on Sensing, Diagnostics, Prognostics, and Control (SDPC) (IEEE, 2019). doi:10.1109/sdpc.2019.00136. DOI

Jumper J. et al. Highly accurate protein structure prediction with AlphaFold. Nature 596, 583–589 (8 2021). PubMed PMC