Segmentation of macromolecular structures is the primary bottleneck for studying biomolecules and their organization with electron microscopy in 2D/3D - requiring months of manual effort. Transformer-based Rapid Dimensionless Instance Segmentation (TARDIS) is a deep learning framework that automatically and accurately annotates membranes and filaments. Pre-trained TARDIS models can segment electron tomography (ET) reconstructions from both 3D and 2D electron micrographs of cryo and plastic-embedded samples. Furthermore, by implementing a novel geometric transformer architecture, TARDIS is the only method to provide accurate instance segmentations of these structures. Reducing the annotation time for ET data from months to minutes, we demonstrate segmentation of membranes and filaments in over 13,000 tomograms in the CZII Data Portal. TARDIS thus enables quantitative biophysical analysis at scale for the first time. We show this in application to kinetochore-microtubule attachment and viral-membrane interactions. TARDIS can be extended to new biomolecules and applications and open-source at https://github.com/SMLC-NYSBC/TARDIS.

Center for Computational Biology Flatiron Institute New York United States

Center for Membrane and Cell Physiology University of Virginia School of Medicine Charlottesville United States

Centre for Bioimaging Sciences Department of Biological Sciences National University of Singapore Singapore Singapore

Core Facility Cellular Imaging Faculty of Medicine Carl Gustav Carus Technische Universität Dresden Dresden Germany

Department of Anesthesiology Columbia University Irving Medical Center New York United States

Department of Cell Biology University of Virginia School of Medicine Charlottesville United States

Department of Chemistry and Biochemistry City College of New York United States

Department of Molecular Physiology and Biological Physics University of Virginia School of Medicine Charlottesville United States

Department of Physiology and Cellular Biophysics Columbia University Irving Medical Center New York United States

Experimental Center Faculty of Medicine Carl Gustav Carus Technische Universität Dresden Dresden Germany

Institute of Biotechnology of the Czech Academy of Sciences BIOCEV Vestec Czech Republic

Programme in Emerging Infectious Diseases Duke National University of Singapore Medical School Singapore Singapore

Simons Electron Microscopy Center New York Structural Biology Center New York United States

Simons Machine Learning Center New York Structural Biology Center New York United States

Structural Biology Initiative CUNY Advanced Science Research Center New York United States

See more in PubMed

Steere R. L. Electron microscopy of structural detail in frozen biological specimens. J. Biophys. Biochem. Cytol. 3, 45–60 (1957). PubMed PMC

Milne J. L. S. et al. Cryo-electron microscopy--a primer for the non-microscopist. FEBS J. 280, 28–45 (2013). PubMed PMC

Mastronarde D. N. Dual-Axis Tomography: An Approach with Alignment Methods That Preserve Resolution. Journal of Structural Biology 120, 343–352 (December 1997). PubMed

McEwen B. F. & Marko M. Chapter 5 Three-Dimensional Transmission Electron Microscopy and Its Application to Mitosis Research. in 81–111 (1998). PubMed

Peddie C. J. et al. Volume electron microscopy. Nat. Rev. Methods Primers 2, 51 (2022). PubMed PMC

Doerr A. Cryo-electron tomography. Nat. Methods 14, 34–34 (2017).

Nogales E. & Mahamid J. Bridging structural and cell biology with cryo-electron microscopy. Nature 628, 47–56 (2024). PubMed PMC

Tollervey F., Rios M. U., Zagoriy E., Woodruff J. B. & Mahamid J. Native molecular architectures of centrosomes in C. elegans embryos. bioRxivorg (2024) doi: 10.1101/2024.04.03.587742. PubMed DOI PMC

Berger C. et al. Cryo-electron tomography on focused ion beam lamellae transforms structural cell biology. Nat. Methods 20, 499–511 (2023). PubMed

Kiewisz R. et al. Three-dimensional structure of kinetochore-fibers in human mitotic spindles. Elife 11, (July 2022). PubMed PMC

Redemann S. et al. C. elegans chromosomes connect to centrosomes by anchoring into the spindle network. Nat. Commun. 8, 15288 (August 2017). PubMed PMC

Yu C.-H. et al. Central-spindle microtubules are strongly coupled to chromosomes during both anaphase A and anaphase B. Molecular Biology of the Cell 30, 2503–2514 (September 2019). PubMed PMC

Xu C., Han W. & Cong Y. Cryo-EM and cryo-ET of the spike, virion, and antibody neutralization of SARS-CoV-2 and VOCs. Curr. Opin. Struct. Biol. 82, 102664 (2023). PubMed

van Hoorn C. & Carter A. P. A cryo-ET study of ciliary rootlet organization. (2024) doi: 10.7554/elife.91642. PubMed DOI PMC

Creekmore B. C., Kixmoeller K., Black B. E., Lee E. B. & Chang Y.-W. Ultrastructure of human brain tissue vitrified from autopsy revealed by cryo-ET with cryo-plasma FIB milling. Nat. Commun. 15, 2660 (2024). PubMed PMC

Guaita M., Watters S. C. & Loerch S. Recent advances and current trends in cryo-electron microscopy. Curr. Opin. Struct. Biol. 77, 102484 (2022). PubMed PMC

Iudin A. et al. EMPIAR: The electron microscopy public image archive. Nucleic Acids Res. 51, D1503–D1511 (2023). PubMed PMC

The Chan Zuckerberg Institute. Cryo-ET Data Portal. Preprint at (2023).

Ermel U. et al. A data portal for providing standardized annotations for cryo-electron tomography. Nat. Methods (2024) doi: 10.1038/s41592-024-02477-2. PubMed DOI

wwPDB Consortium. EMDB-the Electron Microscopy Data Bank. Nucleic Acids Res. 52, D456–D465 (2024). PubMed PMC

Eisenstein F. et al. Parallel cryo electron tomography on in situ lamellae. Nat. Methods 20, 131–138 (2023). PubMed

Kiewisz R., Baum D., Müller-Reichert T. & Fabig G. Serial-section Electron Tomography and Quantitative Analysis of Microtubule Organization in 3D-reconstructed Mitotic Spindles. Bio Protoc. 13, (2023). PubMed PMC

Gudimchuk N. B. et al. Mechanisms of microtubule dynamics and force generation examined with computational modeling and electron cryotomography. Nat. Commun. 11, 1–15 (July 2020). PubMed PMC

McLaren M. et al. CryoEM reveals that ribosomes in microsporidian spores are locked in a dimeric hibernating state. Nat. Microbiol. 8, 1834–1845 (2023). PubMed PMC

Franken L. E. et al. A general mechanism of ribosome dimerization revealed by single-particle cryo-electron microscopy. Nat. Commun. 8, (2017). PubMed PMC

Bepler T. et al. Antiviral activity of the host defense peptide piscidin 1: investigating a membrane-mediated mode of action. Front. Chem. 12, 1379192 (2024). PubMed PMC

Bepler T. et al. Positive-unlabeled convolutional neural networks for particle picking in cryo-electron micrographs. Nat. Methods 16, 1153–1160 (2019). PubMed PMC

Fabig G. et al. Male meiotic spindle features that efficiently segregate paired and lagging chromosomes. eLife 9, (March 2020). PubMed PMC

Redemann S. & Nazockdast E. Integrated 3D Tomography and Computational Modeling to Study Forces in Metaphase Spindles. FASEB J. 34, 1–1 (April 2020).

Mendonça L. et al. Correlative multi-scale cryo-imaging unveils SARS-CoV-2 assembly and egress. Nat. Commun. 12, 4629 (2021). PubMed PMC

Rast A. et al. Biogenic regions of cyanobacterial thylakoids form contact sites with the plasma membrane. Nat. Plants 5, 436–446 (2019). PubMed

Heinrich L. et al. Whole-cell organelle segmentation in volume electron microscopy. Nature 599, 141–146 (2021). PubMed

Zimyanin V. et al. Using 3D large scale tomography to study force generation in the mitotic spindle. Microsc. Microanal. 30, (2024).

Hylton R. K. & Swulius M. T. Challenges and triumphs in cryo-electron tomography. iScience 24, 102959 (2021). PubMed PMC

Schindelin J. et al. Fiji: an open-source platform for biological-image analysis. Nature Methods 9, 676–682 (July 2012). PubMed PMC

Chiu C.-L., Clack N. & the napari community. napari: a Python Multi-Dimensional Image Viewer Platform for the Research Community. in (Microscopy and Methods, 2022).

Kremer J. R., Mastronarde D. N. & McIntosh J. R. Computer Visualization of Three-Dimensional Image Data Using IMOD. Journal of Structural Biology 116, 71–76 (January 1996). PubMed

Martinez-Sanchez A., Garcia I., Asano S., Lucic V. & Fernandez J.-J. Robust membrane detection based on tensor voting for electron tomography. J. Struct. Biol. 186, 49–61 (2014). PubMed

Lorenz L. et al. MemBrain v2: an end-to-end tool for the analysis of membranes in cryo-electron tomography. bioRxivorg (2023).

Stalling D., Westerhoff M. & Hege H.-C. Amira: a Highly Interactive System for Visual Data Analysis. in (eds. Hansen C. D. & Johnson C. R.) 749–767 (Elsevier, 2005).

Inc., C. T. C. Dragonfly. Preprint at (2022).

Archit A. et al. Segment Anything for microscopy. Nat. Methods 1–13 (2025). PubMed PMC

Kiewisz R., Fabig G., Müller-Reichert T. & Bepler T. Automated Segmentation of 3D Cytoskeletal Filaments from Electron Micrographs with TARDIS. Microsc. Microanal. 29, 970–972 (July 2023).

Kiewisz R. & Bepler T. Membrane and microtubule rapid instance segmentation with dimensionless instance segmentation by learning graph representations of point clouds. NeurIPS Machine Learning in Structural Biology Workshop Preprint at (2022).

Lomonossoff G. P. & Wege C. TMV particles: The journey from fundamental studies to bionanotechnology applications. Adv. Virus Res. 102, 149–176 (2018). PubMed PMC

Franklin R. E. Structure of tobacco mosaic virus: Location of the ribonucleic acid in the tobacco mosaic virus particle. Nature 177, 928–930 (1956).

Ishemgulova A., Noble A., Bepler T. & De Marco A. Preparation Of Labeled Cryo-ET Datasets For Training And Evaluation Of Machine Learning Models. in (NeurIPS, 2023).

Conway W., Ha G. & Needleman D. Dual mechanism of kinetochore microtubule detachment. Biophysics (2023).

Tooley J. & Stukenberg P. T. The Ndc80 complex: integrating the kinetochore’s many movements. Chromosome Res. 19, 377–391 (2011). PubMed PMC

Agarwal S. et al. Cdt1 stabilizes kinetochore–microtubule attachments via an Aurora B kinase–dependent mechanism. J. Cell Biol. 217, 3446–3463 (October 2018). PubMed PMC

Godek K. M., Kabeche L. & Compton D. A. Regulation of kinetochore–microtubule attachments through homeostatic control during mitosis. Nat. Rev. Mol. Cell Biol. 16, 57–64 (January 2015). PubMed PMC

Cimini D. et al. Merotelic kinetochore orientation is a major mechanism of aneuploidy in mitotic mammalian tissue cells. J. Cell Biol. 153, 517–527 (2001). PubMed PMC

Ribeiro R. M. et al. Estimation of the initial viral growth rate and basic reproductive number during acute HIV-1 infection. J. Virol. 84, 6096–6102 (2010). PubMed PMC

Bartholomeeusen K. et al. Chikungunya fever. Nat. Rev. Dis. Primers 9, 17 (2023). PubMed PMC

Harrison S. C. Viral membrane fusion. Virology 479-480, 498–507 (2015). PubMed PMC

Roy S. K. & Bhattacharjee S. Dengue virus: epidemiology, biology, and disease aetiology. Can. J. Microbiol. 67, 687–702 (2021). PubMed

Ooi J. S. G. et al. Tomography studies of the dengue fusion process. Acta Crystallogr. A Found. Adv. 79, C566–C566 (August 2023).

Gong B., Johnston J. D., Thiemicke A., de Marco A. & Meyer T. Endoplasmic reticulum-plasma membrane contact gradients direct cell migration. Nature 631, 415–423 (2024). PubMed PMC

Sharma K. D., Doktorova M., Waxham M. N. & Heberle F. A. Cryo-EM images of phase-separated lipid bilayer vesicles analyzed with a machine-learning approach. Biophys. J. 123, 2877–2891 (2024). PubMed PMC

Mehta D. et al. Halogenated cholesterol alters the phase behavior of ternary lipid membranes. J. Phys. Chem. B 129, 671–683 (2025). PubMed

Heberle F. A. & Waxham M. N. Phase separation in model lipid membranes investigated with cryogenic electron microscopy. Methods Enzymol. 700, 189–216 (2024). PubMed

Redemann S. et al. The Segmentation of Microtubules in Electron Tomograms Using Amira. in (ed. Sharp D. J.) vol. 1136 261–278 (Springer Science+ Business Media, 2014). PubMed

Nogales E. & Downing K. H. Tubulin and Microtubule Structures. in 211–225 (Humana Press, January 2009).

Jiang T. & Cheng J. Target recognition based on CNN with LeakyReLU and PReLU activation functions. in 2019 International Conference on Sensing, Diagnostics, Prognostics, and Control (SDPC) (IEEE, 2019). doi: 10.1109/sdpc.2019.00136. DOI

Kiewisz R., Müller-Reichert T. & Fabig G. High-throughput screening of mitotic mammalian cells for electron microscopy using classic histological dyes. in (eds. Müller-Reichert T. & Verkade P.) vol. 162 151–170 (Academic Press Inc., 2021). PubMed

Mastronarde D. N. Automated electron microscope tomography using robust prediction of specimen movements. Journal of Structural Biology 152, 36–51 (October 2005). PubMed

Mastronarde D. N. SerialEM: A Program for Automated Tilt Series Acquisition on Tecnai Microscopes Using Prediction of Specimen Position. Microsc. Microanal. 9, 1182–1183 (August 2003).

Lindow N. et al. Semi automatic stitching of filamentous structures in image stacks from serial section electron tomography. J. Microsc. 284, 25–44 (October 2021). PubMed

Banks E. J. et al. Asymmetric peptidoglycan editing generates cell curvature in Bdellovibrio predatory bacteria. Nat. Commun. 13, 1509 (2022). PubMed PMC

Foster H. E., Ventura Santos C. & Carter A. P. A cryo-ET survey of microtubules and intracellular compartments in mammalian axons. J. Cell Biol. 221, (2022). PubMed PMC

Jumper J. et al. Highly accurate protein structure prediction with AlphaFold. Nature 596, 583–589 (August 2021). PubMed PMC