PURPOSE: The aims of this work are (1) to explore deep learning (DL) architectures, spectroscopic input types, and learning designs toward optimal quantification in MR spectroscopy of simulated pathological spectra; and (2) to demonstrate accuracy and precision of DL predictions in view of inherent bias toward the training distribution. METHODS: Simulated 1D spectra and 2D spectrograms that mimic an extensive range of pathological in vivo conditions are used to train and test 24 different DL architectures. Active learning through altered training and testing data distributions is probed to optimize quantification performance. Ensembles of networks are explored to improve DL robustness and reduce the variance of estimates. A set of scores compares performances of DL predictions and traditional model fitting (MF). RESULTS: Ensembles of heterogeneous networks that combine 1D frequency-domain and 2D time-frequency domain spectrograms as input perform best. Dataset augmentation with active learning can improve performance, but gains are limited. MF is more accurate, although DL appears to be more precise at low SNR. However, this overall improved precision originates from a strong bias for cases with high uncertainty toward the dataset the network has been trained with, tending toward its average value. CONCLUSION: MF mostly performs better compared to the faster DL approach. Potential intrinsic biases on training sets are dangerous in a clinical context that requires the algorithm to be unbiased to outliers (i.e., pathological data). Active learning and ensemble of networks are good strategies to improve prediction performances. However, data quality (sufficient SNR) has proven as a bottleneck for adequate unbiased performance-like in the case of MF.

Department for Biomedical Research University of Bern Bern Switzerland

Department of Biomedical Engineering Brno University of Technology Brno Czech Republic

Department of System Analysis Integrated Assessment and Modelling Data Science for Environmental Research Group EAWAG Dübendorf Switzerland

Graduate School for Cellular and Biomedical Sciences University of Bern Bern Switzerland

Institute of Scientific Instruments of the Czech Academy of Sciences Brno Czech Republic Brno Czech Republic

MR Methodology Department for Diagnostic and Interventional Neuroradiology University of Bern Bern Switzerland

Translational Imaging Center Swiss Institute for Translational and Entrepreneurial Medicine Bern Switzerland

Zobrazit více v PubMed

De Graaf RA. In Vivo NMR Spectroscopy: Principles and Techniques. 3rd ed. John Wiley & Sons; 2019.

Kreis R, Boer V, Choi IY, et al. Terminology and concepts for the characterization of in vivo MR spectroscopy methods and MR spectra: background and experts' consensus recommendations. NMR Biomed. 2020;34:e4347. doi:10.1002/nbm.4347

Near J, Harris AD, Juchem C, et al. Preprocessing, analysis and quantification in single-voxel magnetic resonance spectroscopy: experts' consensus recommendations. NMR Biomed. 2021;34:e4257. doi:10.1002/nbm.4257

Ratiney H, Sdika M, Coenradie Y, Cavassila S, van Ormondt D, Graveron-Demilly D. Time-domain semi-parametric estimation based on a metabolite basis set. NMR Biomed. 2005;18:1-13. doi:10.1002/nbm.895

Provencher SW. Estimation of metabolite concentrations from localized in vivo proton NMR spectra. Magn Reson Med. 1993;30:672-679. doi:10.1002/mrm.1910300604

Wilson M, Reynolds G, Kauppinen RA, Arvanitis TN, Peet AC. A constrained least-squares approach to the automated quantitation of in vivo 1H magnetic resonance spectroscopy data. Magn Reson Med. 2011;65:1-12. doi:10.1002/mrm.22579

Chong DGQ, Kreis R, Bolliger CS, Boesch C, Slotboom J. Two-dimensional linear-combination model fitting of magnetic resonance spectra to define the macromolecule baseline using FiTAID, a fitting tool for arrays of interrelated datasets. MAGMA. 2011;24:147-164. doi:10.1007/s10334-011-0246-y

Bhogal AA, Schür RR, Houtepen LC, et al. 1H-MRS processing parameters affect metabolite quantification: the urgent need for uniform and transparent standardization. NMR Biomed. 2017;30:e3804. doi:10.1002/nbm.3804

Marjańska M, Deelchand DK, Kreis R, et al. Results and interpretation of a fitting challenge for MR spectroscopy set up by the MRS study group of ISMRM. Magn Reson Med. 2022;87:11-32. doi:10.1002/mrm.28942

Lecun Y, Bengio Y, Hinton G. Deep learning. Nature. 2015;521:436-444. doi:10.1038/nature14539

Lundervold AS, Lundervold A. An overview of deep learning in medical imaging focusing on MRI. Z Med Phys. 2019;29:102-127. doi:10.1016/j.zemedi.2018.11.002

Lam F, Li Y, Peng X. Constrained magnetic resonance spectroscopic imaging by learning nonlinear low-dimensional models. IEEE Trans Med Imaging. 2020;39:545-555. doi:10.1109/TMI.2019.2930586

Klukowski P, Augoff M, ZieRba M, Drwal M, Gonczarek A, Walczak MJ. NMRNet: a deep learning approach to automated peak picking of protein NMR spectra. Bioinformatics. 2018;34:2590-1597. doi:10.1093/bioinformatics/bty134

Hatami N, Sdika M, Ratiney H. Magnetic resonance spectroscopy quantification using deep learning. Lect Notes Comput Sci. 2018;11070:467-475. doi:10.1007/978-3-030-00928-1_53

Lee H, Lee HH, Kim H. Reconstruction of spectra from truncated free induction decays by deep learning in proton magnetic resonance spectroscopy. Magn Reson Med. 2020;84:559-568. doi:10.1002/mrm.28164

Iqbal Z, Nguyen D, Thomas MA, Jiang S. Deep learning can accelerate and quantify simulated localized correlated spectroscopy. Sci Rep. 2021;11:8727. doi:10.1038/s41598-021-88158-y

Lee HH, Kim H. Intact metabolite spectrum mining by deep learning in proton magnetic resonance spectroscopy of the brain. Magn Reson Med. 2019;82:33-48. doi:10.1002/mrm.27727

Kyathanahally SP, Döring A, Kreis R. Deep learning approaches for detection and removal of ghosting artifacts in MR spectroscopy. Magn Reson Med. 2018;80:851-863. doi:10.1002/mrm.27096

Gurbani SS, Schreibmann E, Maudsley AA, et al. A convolutional neural network to filter artifacts in spectroscopic MRI. Magn Reson Med. 2018;80:1765-1775. doi:10.1002/mrm.27166

Tapper S, Mikkelsen M, Dewey BE, et al. Frequency and phase correction of J-difference edited MR spectra using deep learning. Magn Reson Med. 2021;85:1755-1765. doi:10.1002/mrm.28525

Jang J, Lee HH, Park JA, Kim H. Unsupervised anomaly detection using generative adversarial networks in 1H-MRS of the brain. J Magn Reson. 2021;325:106936. doi:10.1016/j.jmr.2021.106936

Lee HH, Kim H. Deep learning-based target metabolite isolation and big data-driven measurement uncertainty estimation in proton magnetic resonance spectroscopy of the brain. Magn Reson Med. 2020;84:1689-1706. doi:10.1002/MRM.28234

Gurbani SS, Sheriff S, Maudsley AA, Shim H, Cooper LAD. Incorporation of a spectral model in a convolutional neural network for accelerated spectral fitting. Magn Reson Med. 2019;81:3346-3357. doi:10.1002/mrm.27641

Chandler M, Jenkins C, Shermer SM, Langbein FC. MRSNet: metabolite quantification from edited magnetic resonance spectra with convolutional neural networks. 2019 arXiv:1909.03836v1 [eess.IV]. 10.48550/arXiv.1909.03836

Litjens G, Kooi T, Bejnordi BE, et al. A survey on deep learning in medical image analysis. Med Image Anal. 2017;42:60-88. doi:10.1016/J.MEDIA.2017.07.005

Gyori NG, Palombo M, Clark CA, Zhang H, Alexander DC. Training data distribution significantly impacts the estimation of tissue microstructure with machine learning. Magn Reson Med. 2022;87:932-947. doi:10.1002/MRM.29014

Espi M, Fujimoto M, Kinoshita K, Nakatani T. Exploiting spectro-temporal locality in deep learning based acoustic event detection. J Audio Speech Music Proc. 2015;2015:26. doi:10.1186/s13636-015-0069-2

Thomas S, Ganapathy S, Saon G, Soltau H. Analyzing convolutional neural networks for speech activity detection in mismatched acoustic conditions. In 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP); 2014:2519-2523. 10.1109/ICASSP.2014.6854054

Alaskar H. Deep learning-based model architecture for time-frequency images analysis. Int J Adv Comput Sci Appl. 2018;9:486-494. doi:10.14569/IJACSA.2018.091268

Zagoruyko S, Komodakis N. Wide residual networks. In ArXiv; 2017:arXiv:1605.07146. 10.5244/C.30.87

Cohn DA, Ghahramani Z, Jordan MI. Active learning with statistical models. J Artif Intell Res. 1996;4:129-145. doi:10.1613/JAIR.295

Hansen LK, Salamon P. Neural network ensembles. IEEE Trans Pattern Anal Mach Intell. 1990;12:993-1001. doi:10.1109/34.58871

Krizhevsky A, Sutskever I, Hinton GE. ImageNet classification with deep convolutional neural networks. Commun ACM. 2017;60:84-90. doi:10.1145/3065386

Patra S, Bruzzone L. A cluster-assumption based batch mode active learning technique. Pattern Recognit Lett. 2012;33:1042-1048. doi:10.1016/J.PATREC.2012.01.015

Maiora J, Ayerdi B, Graña M. Random forest active learning for AAA thrombus segmentation in computed tomography angiography images. Neurocomputing. 2014;126:71-77. doi:10.1016/J.NEUCOM.2013.01.051

Kutsuna N, Higaki T, Matsunaga S, et al. Active learning framework with iterative clustering for bioimage classification. Nat Commun. 2012;3:1032. doi:10.1038/ncomms2030

Lewis DD, Gale WA. A sequential algorithm for training text classifiers. In the 17th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, 1994: 3-12. 10.1007/978-1-4471-2099-5_1

Tuia D, Ratle F, Pacifici F, Kanevski MF, Emery WJ. Active learning methods for remote sensing image classification. IEEE Trans Geosci Remote Sens. 2009;48:2218-2232. doi:10.1109/TGRS.2008.2010404

Silva C, Ribeiro B. Margin-based active learning and background knowledge in text mining. In the 4th International Conference on Hybrid Intelligent Systems, 2005: 8-13. 10.1109/ICHIS.2004.70

Pedrosa de Barros N, McKinley R, Wiest R, Slotboom J. Improving labeling efficiency in automatic quality control of MRSI data. Magn Reson Med. 2017;78:2399-2405. doi:10.1002/mrm.26618

Bishop CM. Neural Networks for Pattern Recognition. Oxford University Press; 2005.

Freund Y, Schapire RE. A decision-theoretic generalization of on-line learning and an application to boosting. J Comput Syst Sci. 1997;55:119-139. doi:10.1006/jcss.1997.1504

Chen T, Guestrin C. XGBoost: a scalable tree boosting system. In the 22nd International Conference on Knowledge Discovery and Data Mining; 2016: 785-794. 10.1145/2939672

Ke G, Meng Q, Finley T, et al. LightGBM: a highly efficient gradient boosting decision tree. In the 31st International Conference on Neural Information Processing Systems; 2017: 3149-3157. 10.5555/3294996

Soher BJ, Semanchuk P, Todd D, Steinberg J, Young K. VeSPA: integrated applications for RF pulse design, spectral simulation and MRS data analysis. In Proceedings of the 19th Annual Meeting of ISMRM, Montréal, Québec, Canada. 2011, 1410.

Oz G, Tkac I. Short-echo, single-shot, full-intensity proton magnetic resonance spectroscopy for neurochemical profiling at 4 T: validation in the cerebellum and brainstem. Magn Reson Med. 2011;65:901-910. doi:10.1002/mrm.22708

The Mathworks Inc. MATLAB (R2019a).MathWorks Inc 2019.

Marjańska M, McCarten JR, Hodges J, et al. Region-specific aging of the human brain as evidenced by neurochemical profiles measured noninvasively in the posterior cingulate cortex and the occipital lobe using 1H magnetic resonance spectroscopy at 7 T. Neuroscience. 2017;354:168-177. doi:10.1016/j.neuroscience.2017.04.035

Hoefemann M, Bolliger CS, Chong DGQ, van der Veen JW, Kreis R. Parameterization of metabolite and macromolecule contributions in interrelated MR spectra of human brain using multidimensional modeling. NMR Biomed. 2020;33:e4328. doi:10.1002/nbm.4328

Bottomley PA, Griffiths JR. Handbook of Magnetic Resonance Spectroscopy in Vivo: MRS Theory, Practice and Applications. 1st ed. Hoboken, NJ, Wiley & Sons; 2016.

Oz G, Alger JR, Barker PB, et al. Clinical proton MR spectroscopy in central nervous system disorders. Radiology. 2014;270:658-679.

Träber F, Block W, Lamerichs R, Gieseke J, Schild HH. 1H metabolite relaxation times at 3.0 Tesla: measurements of T1 and T2 values in normal brain and determination of regional differences in transverse relaxation. J Magn Reson Imaging. 2004;19:537-545. doi:10.1002/jmri.20053

An L, Li S, Shen J. Simultaneous determination of metabolite concentrations, T1 and T2 relaxation times. Magn Reson Med. 2017;78:2072-2081.

Zhang Y, Shen J. Simultaneous quantification of glutamate and glutamine by J-modulated spectroscopy at 3 Tesla. Magn Reson Med. 2016;76:725-732.

Cudalbu C, Behar KL, Bhattacharyya PK, et al. Contribution of macromolecules to brain 1H MR spectra: experts' consensus recommendations. NMR Biomed. 2021;34:e4393. doi:10.1002/nbm.4393

Van RG, Drake FL. Python 3 Reference Manual. CreateSpace; 2009.

Gulli A, Pal S. Deep Learning with Keras. Packt Publishing; 2017.

Abadi M, Barham P, Chen J, et al. TensorFlow: a system for large-scale machine learning. In the 12th USENIX Symposium on Operating Systems Design and Implementation; 2016: 265-283. 10.5555/3026877.3026899

Bisong E. Google colaboratory. Building Machine Learning and Deep Learning Models on Google Cloud Platform. Apress, New York City; 2019. doi:10.1007/978-1-4842-4470-8_7

Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition. In arXiv; 2015:1409.1556.

He K, Zhang X, Ren S, Sun J. Deep residual learning for image recognition. In the 2016 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR); 2016: 770-778. 10.1109/CVPR.2016.90

Szegedy C, Vanhoucke V, Ioffe S, Shlens J, Wojna Z. Rethinking the inception architecture for computer vision. In the 2016 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR); 2016: 2818-2826. 10.1109/CVPR.2016.308

Szegedy C, Liu W, Jia Y, et al. Going deeper with convolutions. In 2015 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR); 2015: 1-9. 10.1109/CVPR.2015.7298594

Szegedy C, Ioffe S, Vanhoucke V, Alemi AA. Inception-v4, inception-ResNet and the impact of residual connections on learning. In the 31st AAAI Conference on Artificial Intelligence; 2017: 4278-4284. 10.48550/arXiv.1602.07261

Ioffe S, Szegedy C. Batch normalization: accelerating deep network training by reducing internal covariate shift. In ArXiv; 2015:arXiv:1502.03167.

Clevert DA, Unterthiner T, Hochreiter S. Fast and accurate deep network learning by exponential linear units (ELUs). In: ArXiv; 2016:arXiv:1511.07289.

Snoek J, Larochelle H, Adams RP. Practical Bayesian optimization of machine learning algorithms. In: ArXiv; 2012:arXiv:1206.2944.

Rizzo R, Kreis R. Accounting for bias in estimated metabolite concentrations from cohort studies as caused by limiting the fitting parameter space. In Proceedings of the 2021 ISMRM & SMRT Annual Meeting and Exhibition, Virtual meeting, May 15-20, 2021. p. 2011.

Ronneberger O, Fischer P, Brox T. U-Net: convolutional networks for biomedical image segmentation. Lect Notes Comput Sci. 2015;9351:234-241. doi:10.1007/978-3-319-24574-4_28

Kingma DP, Ba JL. Adam: a method for stochastic optimization. In ArXiv; 2017:arXiv:1412.6980.

Bengio Y. Practical recommendations for gradient-based training of deep architectures. Lect Notes Comput Sci. 2012;7700:437-478.

Bengio Y, Goodfellow IJ, Courville A. Optimization for training deep models. Deep Learning. MIT Press; 2016.

Landheer K, Juchem C. Are Cramér-Rao lower bounds an accurate estimate for standard deviations in in vivo magnetic resonance spectroscopy? NMR Biomed. 2021;34:e4521. doi:10.1002/nbm.4521

Bolliger CS, Boesch C, Kreis R. On the use of Cramér-Rao minimum variance bounds for the design of magnetic resonance spectroscopy experiments. Neuroimage. 2013;83:1031-1040. doi:10.1016/j.neuroimage.2013.07.062

Hong S, Shen J. Impact of training size on deep learning performance in in vivo 1H MRS. In Proceedings of the 2021 ISMRM & SMRT Annual Meeting and Exhibition, Virtual meeting, May 15-20, 2021, p. 2015.

Lee HH, Kim H. Bayesian deep learning-based 1 H-MRS of the brain: metabolite quantification with uncertainty estimation using Monte Carlo dropout. Magn Reson Med. 2022;88:38-52. doi:10.1002/MRM.29214

Rizzo R, Dziadosz M, Kyathanahally SP, Reyes M, Kreis R. Reliability of quantification estimates in MR spectroscopy: CNNs vs traditional model fitting. Med Image Comput Comput Assist Interv-MICCAI 2022 Lect Notes Comput Sci. 2022;13438:715-724. doi:10.1007/978-3-031-16452-1_68

Gal Y. 2016 Uncertainty in deep learning. https://mlg.eng.cam.ac.uk/yarin/thesis/thesis.pdf

Kendall A, Gal Y. What uncertainties do we need in Bayesian deep learning for computer vision? In the 31st Conference on Neural Information Processing Systems (NIPS); 2017.

Sanchez T, Caramiaux B, Thiel P, Mackay WE. Deep learning uncertainty in machine teaching. In 27th Annual Conference on Intelligent User Interfaces (IUI), Vol. 1, 2022. 10.1145/3490099.3511117

Abdar M, Pourpanah F, Hussain S, et al. A review of uncertainty quantification in deep learning: techniques, applications and challenges. Inf Fusion. 2021;76:243-297. doi:10.1016/j.inffus.2021.05.008

Jungo A, Reyes M. Assessing reliability and challenges of uncertainty estimations for medical image segmentation. In International Conference on Medical Image Computing and Computer Assisted Intervention - MICCAI 2019: 22nd International Conference, Proceedings, Part II. Berlin, Heidelberg, Springer-Verlag. 10.1007/978-3-030-32245-8_6

Ennab M, McHeick H. Designing an interpretability-based model to explain the artificial intelligence algorithms in healthcare. Diagnostics. 2022;12:1557. doi:10.3390/DIAGNOSTICS12071557

Meng C, Trinh L, Xu N, Enouen J, Liu Y. Interpretability and fairness evaluation of deep learning models on MIMIC-IV dataset. Sci Rep. 2022;12:1-28. doi:10.1038/s41598-022-11012-2

Ma DJ, Le HAM, Ye Y, et al. MR spectroscopy frequency and phase correction using convolutional neural networks. Magn Reson Med. 2022;87:1700-1710. doi:10.1002/MRM.29103

Shamaei AM, Starcukova J, Pavlova I, Starcuk Z. Model-informed unsupervised deep learning approaches to frequency and phase correction of MRS signals. bioRxiv. 2022. doi:10.1101/2022.06.28.497332

Lin A, Andronesi O, Bogner W, et al. Minimum reporting standards for in vivo magnetic resonance spectroscopy (MRSinMRS): Experts' consensus recommendations. NMR Biomed. 2021;34:e4484. doi:10.1002/nbm.4484