Achieving a reliable and accurate biomedical image segmentation is a long-standing problem. In order to train or adapt the segmentation methods or measure their performance, reference segmentation masks are required. Usually gold-standard annotations, i.e. human-origin reference annotations, are used as reference although they are very hard to obtain. The increasing size of the acquired image data, large dimensionality such as 3D or 3D + time, limited human expert time, and annotator variability, typically result in sparsely annotated gold-standard datasets. Reliable silver-standard annotations, i.e. computer-origin reference annotations, are needed to provide dense segmentation annotations by fusing multiple computer-origin segmentation results. The produced dense silver-standard annotations can then be either used as reference annotations directly, or converted into gold-standard ones with much lighter manual curation, which saves experts' time significantly. We propose a novel full-resolution multi-rater fusion convolutional neural network (CNN) architecture for biomedical image segmentation masks, called DeepFuse, which lacks any down-sampling layers. Staying everywhere at the full resolution enables DeepFuse to fully benefit from the enormous feature extraction capabilities of CNNs. DeepFuse outperforms the popular and commonly used fusion methods, STAPLE, SIMPLE and other majority-voting-based approaches with statistical significance on a wide range of benchmark datasets as demonstrated on examples of a challenging task of 2D and 3D cell and cell nuclei instance segmentation for a wide range of microscopy modalities, magnifications, cell shapes and densities. A remarkable feature of the proposed method is that it can apply specialized post-processing to the segmentation masks of each rater separately and recover under-segmented object parts during the refinement phase even if the majority of inputs vote otherwise. Thus, DeepFuse takes a big step towards obtaining fast and reliable computer-origin segmentation annotations for biomedical images.
- MeSH
- Humans MeSH
- Neural Networks, Computer * MeSH
- Image Processing, Computer-Assisted * methods MeSH
- Check Tag
- Humans MeSH
- Publication type
- Journal Article MeSH
OBJECTIVES: Artificial Intelligence (AI), particularly deep learning, has significantly impacted healthcare, including dentistry, by improving diagnostics, treatment planning, and prognosis prediction. This systematic mapping review explores the current applications of deep learning in dentistry, offering a comprehensive overview of trends, models, and their clinical significance. MATERIALS AND METHODS: Following a structured methodology, relevant studies published from January 2012 to September 2023 were identified through database searches in PubMed, Scopus, and Embase. Key data, including clinical purpose, deep learning tasks, model architectures, and data modalities, were extracted for qualitative synthesis. RESULTS: From 21,242 screened studies, 1,007 were included. Of these, 63.5% targeted diagnostic tasks, primarily with convolutional neural networks (CNNs). Classification (43.7%) and segmentation (22.9%) were the main methods, and imaging data-such as cone-beam computed tomography and orthopantomograms-were used in 84.4% of cases. Most studies (95.2%) applied fully supervised learning, emphasizing the need for annotated data. Pathology (21.5%), radiology (17.5%), and orthodontics (10.2%) were prominent fields, with 24.9% of studies relating to more than one specialty. CONCLUSION: This review explores the advancements in deep learning in dentistry, particulary for diagnostics, and identifies areas for further improvement. While CNNs have been used successfully, it is essential to explore emerging model architectures, learning approaches, and ways to obtain diverse and reliable data. Furthermore, fostering trust among all stakeholders by advancing explainable AI and addressing ethical considerations is crucial for transitioning AI from research to clinical practice. CLINICAL RELEVANCE: This review offers a comprehensive overview of a decade of deep learning in dentistry, showcasing its significant growth in recent years. By mapping its key applications and identifying research trends, it provides a valuable guide for future studies and highlights emerging opportunities for advancing AI-driven dental care.
- MeSH
- Deep Learning * MeSH
- Humans MeSH
- Dentistry * MeSH
- Check Tag
- Humans MeSH
- Publication type
- Journal Article MeSH
- Review MeSH
- Systematic Review MeSH
OBJECTIVE: The aim of this work was to assemble a large annotated dataset of bitewing radiographs and to use convolutional neural networks to automate the detection of dental caries in bitewing radiographs with human-level performance. MATERIALS AND METHODS: A dataset of 3989 bitewing radiographs was created, and 7257 carious lesions were annotated using minimal bounding boxes. The dataset was then divided into 3 parts for the training (70%), validation (15%), and testing (15%) of multiple object detection convolutional neural networks (CNN). The tested CNN architectures included YOLOv5, Faster R-CNN, RetinaNet, and EfficientDet. To further improve the detection performance, model ensembling was used, and nested predictions were removed during post-processing. The models were compared in terms of the [Formula: see text] score and average precision (AP) with various thresholds of the intersection over union (IoU). RESULTS: The twelve tested architectures had [Formula: see text] scores of 0.72-0.76. Their performance was improved by ensembling which increased the [Formula: see text] score to 0.79-0.80. The best-performing ensemble detected caries with the precision of 0.83, recall of 0.77, [Formula: see text], and AP of 0.86 at IoU=0.5. Small carious lesions were predicted with slightly lower accuracy (AP 0.82) than medium or large lesions (AP 0.88). CONCLUSIONS: The trained ensemble of object detection CNNs detected caries with satisfactory accuracy and performed at least as well as experienced dentists (see companion paper, Part II). The performance on small lesions was likely limited by inconsistencies in the training dataset. CLINICAL SIGNIFICANCE: Caries can be automatically detected using convolutional neural networks. However, detecting incipient carious lesions remains challenging.
- MeSH
- Deep Learning * MeSH
- Humans MeSH
- Dental Caries Susceptibility MeSH
- Neural Networks, Computer MeSH
- Dental Caries * diagnostic imaging MeSH
- Check Tag
- Humans MeSH
- Publication type
- Journal Article MeSH
BACKGROUND: Mean diffusivity (MD) and fractional anisotropy (FA) from diffusion MRI (dMRI) have been associated with cell density and tissue anisotropy across tumors, but it is unknown whether these associations persist at the microscopic level. PURPOSE: To quantify the degree to which cell density and anisotropy, as determined from histology, account for the intra-tumor variability of MD and FA in meningioma tumors. Furthermore, to clarify whether other histological features account for additional intra-tumor variability of dMRI parameters. MATERIALS AND METHODS: We performed ex-vivo dMRI at 200 μm isotropic resolution and histological imaging of 16 excised meningioma tumor samples. Diffusion tensor imaging (DTI) was used to map MD and FA, as well as the in-plane FA (FAIP). Histology images were analyzed in terms of cell nuclei density (CD) and structure anisotropy (SA; obtained from structure tensor analysis) and were used separately in a regression analysis to predict MD and FAIP, respectively. A convolutional neural network (CNN) was also trained to predict the dMRI parameters from histology patches. The association between MRI and histology was analyzed in terms of out-of-sample (R2OS) on the intra-tumor level and within-sample R2 across tumors. Regions where the dMRI parameters were poorly predicted from histology were analyzed to identify features apart from CD and SA that could influence MD and FAIP, respectively. RESULTS: Cell density assessed by histology poorly explained intra-tumor variability of MD at the mesoscopic level (200 μm), as median R2OS = 0.04 (interquartile range 0.01-0.26). Structure anisotropy explained more of the variation in FAIP (median R2OS = 0.31, 0.20-0.42). Samples with low R2OS for FAIP exhibited low variations throughout the samples and thus low explainable variability, however, this was not the case for MD. Across tumors, CD and SA were clearly associated with MD (R2 = 0.60) and FAIP (R2 = 0.81), respectively. In 37% of the samples (6 out of 16), cell density did not explain intra-tumor variability of MD when compared to the degree explained by the CNN. Tumor vascularization, psammoma bodies, microcysts, and tissue cohesivity were associated with bias in MD prediction based solely on CD. Our results support that FAIP is high in the presence of elongated and aligned cell structures, but low otherwise. CONCLUSION: Cell density and structure anisotropy account for variability in MD and FAIP across tumors but cell density does not explain MD variations within the tumor, which means that low or high values of MD locally may not always reflect high or low tumor cell density. Features beyond cell density need to be considered when interpreting MD.
- MeSH
- Anisotropy MeSH
- Diffusion Magnetic Resonance Imaging methods MeSH
- Humans MeSH
- Meningeal Neoplasms * pathology MeSH
- Meningioma * diagnostic imaging pathology MeSH
- Diffusion Tensor Imaging methods MeSH
- Check Tag
- Humans MeSH
- Publication type
- Journal Article MeSH
- Research Support, Non-U.S. Gov't MeSH
- Research Support, N.I.H., Extramural MeSH
In this work, we classify chemotherapeutic agents (topoisomerase inhibitors) based on their effect on U-2 OS cells. We use phase-contrast microscopy images, which are faster and easier to obtain than fluorescence images and support live cell imaging. We use a convolutional neural network (CNN) trained end-to-end directly on the input images without requiring for manual segmentations or any other auxiliary data. Our method can distinguish between tested cytotoxic drugs with an accuracy of 98%, provided that their mechanism of action differs, outperforming previous work. The results are even better when substance-specific concentrations are used. We show the benefit of sharing the extracted features over all classes (drugs). Finally, a 2D visualization of these features reveals clusters, which correspond well to known class labels, suggesting the possible use of our methodology for drug discovery application in analyzing new, unseen drugs.
Parkinson's disease dysgraphia (PDYS), one of the earliest signs of Parkinson's disease (PD), has been researched as a promising biomarker of PD and as the target of a noninvasive and inexpensive approach to monitoring the progress of the disease. However, although several approaches to supportive PDYS diagnosis have been proposed (mainly based on handcrafted features (HF) extracted from online handwriting or the utilization of deep neural networks), it remains unclear which approach provides the highest discrimination power and how these approaches can be transferred between different datasets and languages. This study aims to compare classification performance based on two types of features: features automatically extracted by a pretrained convolutional neural network (CNN) and HF designed by human experts. Both approaches are evaluated on a multilingual dataset collected from 143 PD patients and 151 healthy controls in the Czech Republic, United States, Colombia, and Hungary. The subjects performed the spiral drawing task (SDT; a language-independent task) and the sentence writing task (SWT; a language-dependent task). Models based on logistic regression and gradient boosting were trained in several scenarios, specifically single language (SL), leave one language out (LOLO), and all languages combined (ALC). We found that the HF slightly outperformed the CNN-extracted features in all considered evaluation scenarios for the SWT. In detail, the following balanced accuracy (BACC) scores were achieved: SL-0.65 (HF), 0.58 (CNN); LOLO-0.65 (HF), 0.57 (CNN); and ALC-0.69 (HF), 0.66 (CNN). However, in the case of the SDT, features extracted by a CNN provided competitive results: SL-0.66 (HF), 0.62 (CNN); LOLO-0.56 (HF), 0.54 (CNN); and ALC-0.60 (HF), 0.60 (CNN). In summary, regarding the SWT, the HF outperformed the CNN-extracted features over 6% (mean BACC of 0.66 for HF, and 0.60 for CNN). In the case of the SDT, both feature sets provided almost identical classification performance (mean BACC of 0.60 for HF, and 0.58 for CNN).
- Publication type
- Journal Article MeSH
The complex shape of embryonic cartilage represents a true challenge for phenotyping and basic understanding of skeletal development. X-ray computed microtomography (μCT) enables inspecting relevant tissues in all three dimensions; however, most 3D models are still created by manual segmentation, which is a time-consuming and tedious task. In this work, we utilised a convolutional neural network (CNN) to automatically segment the most complex cartilaginous system represented by the developing nasal capsule. The main challenges of this task stem from the large size of the image data (over a thousand pixels in each dimension) and a relatively small training database, including genetically modified mouse embryos, where the phenotype of the analysed structures differs from the norm. We propose a CNN-based segmentation model optimised for the large image size that we trained using a unique manually annotated database. The segmentation model was able to segment the cartilaginous nasal capsule with a median accuracy of 84.44% (Dice coefficient). The time necessary for segmentation of new samples shortened from approximately 8 h needed for manual segmentation to mere 130 s per sample. This will greatly accelerate the throughput of μCT analysis of cartilaginous skeletal elements in animal models of developmental diseases.
- MeSH
- Cartilage diagnostic imaging MeSH
- Deep Learning * MeSH
- Mice MeSH
- Neural Networks, Computer MeSH
- Image Processing, Computer-Assisted methods MeSH
- X-Rays MeSH
- Developmental Biology MeSH
- Animals MeSH
- Check Tag
- Mice MeSH
- Animals MeSH
- Publication type
- Journal Article MeSH
Objective.Functional specialization is fundamental to neural information processing. Here, we study whether and how functional specialization emerges in artificial deep convolutional neural networks (CNNs) during a brain-computer interfacing (BCI) task.Approach.We trained CNNs to predict hand movement speed from intracranial electroencephalography (iEEG) and delineated how units across the different CNN hidden layers learned to represent the iEEG signal.Main results.We show that distinct, functionally interpretable neural populations emerged as a result of the training process. While some units became sensitive to either iEEG amplitude or phase, others showed bimodal behavior with significant sensitivity to both features. Pruning of highly sensitive units resulted in a steep drop of decoding accuracy not observed for pruning of less sensitive units, highlighting the functional relevance of the amplitude- and phase-specialized populations.Significance.We anticipate that emergent functional specialization as uncovered here will become a key concept in research towards interpretable deep learning for neuroscience and BCI applications.
Schizophrenia is a severe neuropsychiatric disease whose diagnosis, unfortunately, lacks an objective diagnostic tool supporting a thorough psychiatric examination of the patient. We took advantage of today's computational abilities, structural magnetic resonance imaging, and modern machine learning methods, such as stacked autoencoders (SAE) and 3D convolutional neural networks (3D CNN), to teach them to classify 52 patients with schizophrenia and 52 healthy controls. The main aim of this study was to explore whether complex feature extraction methods can help improve the accuracy of deep learning-based classifiers compared to minimally preprocessed data. Our experiments employed three commonly used preprocessing steps to extract three different feature types. They included voxel-based morphometry, deformation-based morphometry, and simple spatial normalization of brain tissue. In addition to classifier models, features and their combination, other model parameters such as network depth, number of neurons, number of convolutional filters, and input data size were also investigated. Autoencoders were trained on feature pools of 1000 and 5000 voxels selected by Mann-Whitney tests, and 3D CNNs were trained on whole images. The most successful model architecture (autoencoders) achieved the highest average accuracy of 69.62% (sensitivity 68.85%, specificity 70.38%). The results of all experiments were statistically compared (the Mann-Whitney test). In conclusion, SAE outperformed 3D CNN, while preprocessing using VBM helped SAE improve the results.
- Publication type
- Journal Article MeSH
BACKGROUND: The recent big data revolution in Genomics, coupled with the emergence of Deep Learning as a set of powerful machine learning methods, has shifted the standard practices of machine learning for Genomics. Even though Deep Learning methods such as Convolutional Neural Networks (CNNs) and Recurrent Neural Networks (RNNs) are becoming widespread in Genomics, developing and training such models is outside the ability of most researchers in the field. RESULTS: Here we present ENNGene-Easy Neural Network model building tool for Genomics. This tool simplifies training of custom CNN or hybrid CNN-RNN models on genomic data via an easy-to-use Graphical User Interface. ENNGene allows multiple input branches, including sequence, evolutionary conservation, and secondary structure, and performs all the necessary preprocessing steps, allowing simple input such as genomic coordinates. The network architecture is selected and fully customized by the user, from the number and types of the layers to each layer's precise set-up. ENNGene then deals with all steps of training and evaluation of the model, exporting valuable metrics such as multi-class ROC and precision-recall curve plots or TensorBoard log files. To facilitate interpretation of the predicted results, we deploy Integrated Gradients, providing the user with a graphical representation of an attribution level of each input position. To showcase the usage of ENNGene, we train multiple models on the RBP24 dataset, quickly reaching the state of the art while improving the performance on more than half of the proteins by including the evolutionary conservation score and tuning the network per protein. CONCLUSIONS: As the role of DL in big data analysis in the near future is indisputable, it is important to make it available for a broader range of researchers. We believe that an easy-to-use tool such as ENNGene can allow Genomics researchers without a background in Computational Sciences to harness the power of DL to gain better insights into and extract important information from the large amounts of data available in the field.
- MeSH
- Genomics MeSH
- Neural Networks, Computer * MeSH
- Protein Structure, Secondary MeSH
- Machine Learning * MeSH
- Publication type
- Journal Article MeSH
