We examined how penalized linear discriminant analysis with resampling, which is a supervised, multivariate, whole-brain reduction technique, can help schizophrenia diagnostics and research. In an experiment with magnetic resonance brain images of 52 first-episode schizophrenia patients and 52 healthy controls, this method allowed us to select brain areas relevant to schizophrenia, such as the left prefrontal cortex, the anterior cingulum, the right anterior insula, the thalamus, and the hippocampus. Nevertheless, the classification performance based on such reduced data was not significantly better than the classification of data reduced by mass univariate selection using a t-test or unsupervised multivariate reduction using principal component analysis. Moreover, we found no important influence of the type of imaging features, namely local deformations or gray matter volumes, and the classification method, specifically linear discriminant analysis or linear support vector machines, on the classification results. However, we ascertained significant effect of a cross-validation setting on classification performance as classification results were overestimated even though the resampling was performed during the selection of brain imaging features. Therefore, it is critically important to perform cross-validation in all steps of the analysis (not only during classification) in case there is no external validation set to avoid optimistically biasing the results of classification studies.

Behavioural and Social Neuroscience Group CEITEC Central European Institute of Technology Masaryk UniversityBrno Czech Republic; Department of Psychiatry University Hospital Brno and Masaryk UniversityBrno Czech Republic

Department of Biomedical Engineering King's College London London UK

Institute of Biostatistics and Analyses Faculty of Medicine Masaryk University Brno Czech Republic

Zobrazit více v PubMed

Andreasen N. C. (1995). Symptoms, signs, and diagnosis of schizophrenia. Lancet 346, 477–481. 10.1016/S0140-6736(95)91325-4 PubMed DOI

Antonova E., Sharma T., Morris R., Kumari V. (2004). The relationship between brain structure and neurocognition in schizophrenia: a selective review. Schizophr. Res. 70, 117–145. 10.1016/j.schres.2003.12.002 PubMed DOI

Ardekani B. A., Tabesh A., Sevy S., Robinson D. G., Bilder R. M., Szeszko P. R. (2011). Diffusion tensor imaging reliably differentiates patients with schizophrenia from healthy volunteers. Hum. Brain Mapp. 32, 1–9. 10.1002/hbm.20995 PubMed DOI PMC

Borgwardt S., Koutsouleris N., Aston J., Studerus E., Smieskova R., Riecher-Röessler A., et al. . (2012). Distinguishing prodromal from first-episode psychosis using neuroanatomical single-subject pattern recognition. Schizophr. Bull. 39, 1105–1114. 10.1093/schbul/sbs095 PubMed DOI PMC

Bunea F., She Y., Ombao H., Gongvatana A., Devlin K., Cohen R. (2011). Penalized least squares regression methods and applications to neuroimaging. Neuroimage 55, 1519–1527. 10.1016/j.neuroimage.2010.12.028 PubMed DOI PMC

Button K. S., Ioannidis J. P. A., Mokrysz C., Nosek B. A., Flint J., Robinson E. S. J., et al. . (2013). Power failure: why small sample size undermines the reliability of neuroscience. Nat. Rev. Neurosci. 14, 365–376. 10.1038/nrn3475 PubMed DOI

Castro E., Gómez-Verdejo V., Martínez-Ramón M., Kiehl K. A., Calhoun V. D. (2014). A multiple kernel learning approach to perform classification of groups from complex-valued fMRI data analysis: application to schizophrenia. Neuroimage 87, 1–17. 10.1016/j.neuroimage.2013.10.065 PubMed DOI PMC

Demirci O., Clark V. P., Magnotta V. A., Andreasen N. C., Lauriello J., Kiehl K. A., et al. . (2008). A review of challenges in the use of fMRI for disease classification/characterization and a projection pursuit application from a multi-site fMRI schizophrenia study. Brain Imaging Behav. 2, 207–226. 10.1007/s11682-008-9028-1 PubMed DOI PMC

Diaz G., Romero E., Hernández-Tamames J. A., Molina V., Malpica N. (2010). Automatic classification of structural MRI for diagnosis of neurodegenerative diseases. Acta Biol. Colomb. 15, 165–180.

Dluhos P., Schwarz D., Kasparek T. (2014). Wavelet features for recognition of first episode of schizophrenia from MRI brain images. Radioengineering 23, 274–281. Available online at: http://www.radioeng.cz/fulltexts/2014/14_01_0274_0281.pdf

Ellison-Wright I., Glahn D. C., Laird A. R., Thelen S. M., Bullmore E. (2008). The anatomy of first-episode and chronic schizophrenia: an anatomical likelihood estimation meta-analysis. Am. J. Psychiatry 165, 1015–1023. 10.1176/appi.ajp.2008.07101562 PubMed DOI PMC

Fan Y., Shen D. G., Davatzikos C. (2005). Classification of structural images via high-dimensional image warping, robust feature extraction, and SVM, in Medical Image Computing and Computer-Assisted Intervention - Miccai 2005, eds Duncan J. S., Gerig G. (Berlin: Springer-Verlag Berlin; ), 1–8. PubMed

Fan Y., Shen D., Gur R. C., Gur R. E., Davatzikos C. (2007). COMPARE: classification of morphological patterns using adaptive regional elements. IEEE Trans. Med. Imaging 26, 93–105. 10.1109/TMI.2006.886812 PubMed DOI

Goldman-Rakic P. S., Selemon L. D. (1997). Functional and anatomical aspects of prefrontal pathology in schizophrenia. Schizophr. Bull. 23, 437–458. 10.1093/schbul/23.3.437 PubMed DOI

Good C. D., Johnsrude I. S., Ashburner J., Henson R. N. A., Friston K. J., Frackowiak R. S. J. (2001). A voxel-based morphometric study of ageing in 465 normal adult human brains. Neuroimage 14, 21–36. 10.1006/nimg.2001.0786 PubMed DOI

Guyon I., Elisseeff A. (2003). An introduction to variable and feature selection. J. Mach. Learn. Res. 3, 1157–1182. Available online at: http://www.jmlr.org/papers/volume3/guyon03a/guyon03a.pdf

Haijma S. V., Van Haren N., Cahn W., Koolschijn P. C., Hulshoff Pol H. E., Kahn R. S. (2013). Brain volumes in schizophrenia: a meta-analysis in over 18 000 subjects. Schizophr. Bull. 39, 1129–1138. 10.1093/schbul/sbs118 PubMed DOI PMC

Ho B.-C., Andreasen N. C., Ziebell S., Pierson R., Magnotta V. (2011). Long-term antipsychotic treatment and brain volumes: a longitudinal study of first-episode schizophrenia. Arch. Gen. Psychiatry 68, 128–137. 10.1001/archgenpsychiatry.2010.199 PubMed DOI PMC

Honea R., Crow T. J., Passingham D., Mackay C. E. (2005). Regional deficits in brain volume in schizophrenia: a meta-analysis of voxel-based morphometry studies. Am. J. Psychiatry 162, 2233–2245. 10.1176/appi.ajp.162.12.2233 PubMed DOI

Janousova E., Schwarz D., Kasparek T. (2015). Combining various types of classifiers and features extracted from magnetic resonance imaging data in schizophrenia recognition. Psychiatry Res. Neuroimaging 232, 237–249. 10.1016/j.pscychresns.2015.03.004 PubMed DOI

Kambeitz J., Kambeitz-Ilankovic L., Leucht S., Wood S., Davatzikos C., Malchow B., et al. . (2015). Detecting neuroimaging biomarkers for schizophrenia: a meta-analysis of multivariate pattern recognition studies. Neuropsychopharmacology 40, 1742–1751. 10.1038/npp.2015.22 PubMed DOI PMC

Karageorgiou E., Schulz S. C., Gollub R. L., Andreasen N. C., Ho B.-C., Lauriello J., et al. . (2011). Neuropsychological testing and structural magnetic resonance imaging as diagnostic biomarkers early in the course of schizophrenia and related psychoses. Neuroinformatics 9, 321–333. 10.1007/s12021-010-9094-6 PubMed DOI PMC

Kasparek T., Thomaz C. E., Sato J. R., Schwarz D., Janousova E., Marecek R., et al. . (2011). Maximum-uncertainty linear discrimination analysis of first-episode schizophrenia subjects. Psychiatry Res. Neuroimaging 191, 174–181. 10.1016/j.pscychresns.2010.09.016 PubMed DOI

Kawasaki Y., Suzuki M., Kherif F., Takahashi T., Zhou S.-Y., Nakamura K., et al. . (2007). Multivariate voxel-based morphometry successfully differentiates schizophrenia patients from healthy controls. Neuroimage 34, 235–242. 10.1016/j.neuroimage.2006.08.018 PubMed DOI

Klöppel S., Abdulkadir A., Jack C. R., Koutsouleris N., Mourao-Miranda J., Vemuri P. (2012). Diagnostic neuroimaging across diseases. Neuroimage 61, 457–463. 10.1016/j.neuroimage.2011.11.002 PubMed DOI PMC

Kovalev V. A., Petrou M., Suckling J. (2003). Detection of structural differences between the brains of schizophrenic patients and controls. Psychiatry Res. Neuroimaging 124, 177–189. 10.1016/S0925-4927(03)00070-2 PubMed DOI

Lemm S., Blankertz B., Dickhaus T., Mueller K.-R. (2011). Introduction to machine learning for brain imaging. Neuroimage 56, 387–399. 10.1016/j.neuroimage.2010.11.004 PubMed DOI

Leonard C. M., Kuldau J. M., Breier J. I., Zuffante P. A., Gautier E. R., Heron D. C., et al. . (1999). Cumulative effect of anatomical risk factors for schizophrenia: an MRI study. Biol. Psychiatry 46, 374–382. 10.1016/S0006-3223(99)00052-9 PubMed DOI

Liddle P. F. (1987). The symptoms of chronic schizophrenia. a re-examination of the positive-negative dichotomy. Br. J. Psychiatry 151, 145–151. 10.1192/bjp.151.2.145 PubMed DOI

Marquand A. F., Rezek I., Buitelaar J., Beckmann C. F. (2016). Understanding heterogeneity in clinical cohorts using normative models: beyond case-control studies. Biol. Psychiatry. 10.1016/j.biopsych.2015.12.023. [Epub ahead of print]. PubMed DOI PMC

Meinshausen N., Bühlmann P. (2010). Stability selection. J. R. Stat. Soc. B-Stat. Methodol. 72, 417–473. 10.1111/j.1467-9868.2010.00740.x DOI

Melonakos E. D., Shenton M. E., Rathi Y., Terry D. P., Bouix S., Kubicki M. (2011). Voxel-based morphometry (VBM) studies in schizophrenia-can white matter changes be reliably detected with VBM? Psychiatry Res. 193, 65–70. 10.1016/j.pscychresns.2011.01.009 PubMed DOI PMC

Michael A. M., Calhoun V. D., Andreasen N. C., Baum S. A. (2008). A method to classify schizophrenia using inter-task spatial correlations of functional brain images, in 2008 30th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, Vol. 1-8 (New York, NY: IEEE; ), 5510–5513. PubMed

Mourao-Miranda J., Reinders A. A. T. S., Rocha-Rego V., Lappin J., Rondina J., Morgan C., et al. . (2012). Individualized prediction of illness course at the first psychotic episode: a support vector machine MRI study. Psychol. Med. 42, 1037–1047. 10.1017/S0033291711002005 PubMed DOI PMC

Nakamura K., Kawasaki Y., Suzuki M., Hagino H., Kurokawa K., Takahashi T., et al. . (2004). Multiple structural brain measures obtained by three-dimensional magnetic resonance imaging to distinguish between schizophrenia patients and normal subjects. Schizophr. Bull. 30, 393–404. 10.1093/oxfordjournals.schbul.a007087 PubMed DOI

Nasrallah H., Schwarzkopf S., Olson S., Coffman J. (1990). Gender differences in schizophrenia on MRI brain-scans. Schizophr. Bull. 16, 205–209. 10.1093/schbul/16.2.205 PubMed DOI

Nieuwenhuis M., van Haren N. E. M., Pol H. E. H., Cahn W., Kahn R. S., Schnack H. G. (2012). Classification of schizophrenia patients and healthy controls from structural MRI scans in two large independent samples. Neuroimage 61, 606–612. 10.1016/j.neuroimage.2012.03.079 PubMed DOI

Niznikiewicz M. A., Kubicki M., Shenton M. E. (2003). Recent structural and functional imaging findings in schizophrenia. Curr. Opin. Psychiatry 16, 123–147. 10.1097/00001504-200303000-00002 DOI

Ota M., Sato N., Ishikawa M., Hori H., Sasayama D., Hattori K., et al. . (2012). Discrimination of female schizophrenia patients from healthy women using multiple structural brain measures obtained with voxel-based morphometry. Psychiatry Clin. Neurosci. 66, 611–617. 10.1111/j.1440-1819.2012.02397.x PubMed DOI

Perkins D. O., Gu H. B., Boteva K., Lieberman J. A. (2005). Relationship between duration of untreated psychosis and outcome in first-episode schizophrenia: a critical review and meta-analysis. Am. J. Psychiatry 162, 1785–1804. 10.1176/appi.ajp.162.10.1785 PubMed DOI

Pettersson-Yeo W., Benetti S., Marquand A. F., Dell'Acqua F., Williams S. C. R., Allen P., et al. . (2013). Using genetic, cognitive and multi-modal neuroimaging data to identify ultra-high-risk and first-episode psychosis at the individual level. Psychol. Med. 43, 2547–2562. 10.1017/S003329171300024X PubMed DOI PMC

Plis S. M., Hjelm D. R., Salakhutdinov R., Allen E. A., Bockholt H. J., Long J. D., et al. . (2014). Deep learning for neuroimaging: a validation study. Front. Neurosci. 8:229. 10.3389/fnins.2014.00229 PubMed DOI PMC

Pohl K. M., Sabuncu M. R. (2009). A unified framework for mr based disease classification, in Information Processing in Medical Imaging, Proceedings, eds Prince J. L., Pham D. L., Myers K. J. (Berlin: Springer-Verlag Berlin; ), 300–313. PubMed PMC

Rosa M. J., Portugal L., Hahn T., Fallgatter A. J., Garrido M. I., Shawe-Taylor J., et al. . (2015). Sparse network-based models for patient classification using fMRI. Neuroimage 105, 493–506. 10.1016/j.neuroimage.2014.11.021 PubMed DOI PMC

Ryali S., Supekar K., Abrams D. A., Menon V. (2010). Sparse logistic regression for whole-brain classification of fMRI data. Neuroimage 51, 752–764. 10.1016/j.neuroimage.2010.02.040 PubMed DOI PMC

Schnack H. G., Kahn R. S. (2016). Detecting neuroimaging biomarkers for psychiatric disorders: sample size matters. Front. Psychiatry 7:50. 10.3389/fpsyt.2016.00050 PubMed DOI PMC

Schwarz D., Kasparek T., Provaznik I., Jarkovsky J. (2007). A deformable registration method for automated morphometry of MRI brain images in neuropsychiatric research. IEEE Trans. Med. Imaging 26, 452–461. 10.1109/TMI.2007.892512 PubMed DOI

Shenton M. E., Dickey C. C., Frumin M., McCarley R. W. (2001). A review of MRI findings in schizophrenia. Schizophr. Res. 49, 1–52. 10.1016/S0920-9964(01)00163-3 PubMed DOI PMC

Silveira C., Marques-Teixeira J., de Bastos-Leite A. J. (2012). More than one century of schizophrenia: an evolving perspective. J. Nerv. Ment. Dis. 200, 1054–1057. 10.1097/NMD.0b013e318275d249 PubMed DOI

Simeone J. C., Ward A. J., Rotella P., Collins J., Windisch R. (2015). An evaluation of variation in published estimates of schizophrenia prevalence from 1990-2013: a systematic literature review. BMC Psychiatry 15:193. 10.1186/s12888-015-0578-7 PubMed DOI PMC

Sun D., van Erp T. G. M., Thompson P. M., Bearden C. E., Daley M., Kushan L., et al. . (2009). Elucidating a magnetic resonance imaging-based neuroanatomic biomarker for psychosis: classification analysis using probabilistic brain atlas and machine learning algorithms. Biol. Psychiatry 66, 1055–1060. 10.1016/j.biopsych.2009.07.019 PubMed DOI PMC

Takayanagi Y., Kawasaki Y., Nakamura K., Takahashi T., Orikabe L., Toyoda E., et al. . (2010). Differentiation of first-episode schizophrenia patients from healthy controls using ROI-based multiple structural brain variables. Prog. Neuro-Psychopharmacol. Biol. Psychiatry 34, 10–17. 10.1016/j.pnpbp.2009.09.004 PubMed DOI

Thomaz C. E., Boardman J. P., Counsell S., Hill D. L. G., Hajnal J. V., Edwards A. D., et al. (2007). A multivariate statistical analysis of the developing human brain in preterm infants. Image Vis. Comput. 25, 981–994. 10.1016/j.imavis.2006.07.011 DOI

Vounou M., Janousova E., Wolz R., Stein J. L., Thompson P. M., Rueckert D., et al. . (2012). Sparse reduced-rank regression detects genetic associations with voxel-wise longitudinal phenotypes in Alzheimer's disease. Neuroimage 60, 700–716. 10.1016/j.neuroimage.2011.12.029 PubMed DOI PMC

Witten D. M., Tibshirani R. (2011). Penalized classification using Fisher's linear discriminant. J. R. Stat. Soc. Ser. B-Stat. Methodol. 73, 753–772. 10.1111/j.1467-9868.2011.00783.x PubMed DOI PMC

Wolfers T., Buitelaar J. K., Beckmann C. F., Franke B., Marquand A. F. (2015). From estimating activation locality to predicting disorder: a review of pattern recognition for neuroimaging-based psychiatric diagnostics. Neurosci. Biobehav. Rev. 57, 328–349. 10.1016/j.neubiorev.2015.08.001 PubMed DOI

Wright I. C., Rabe-Hesketh S., Woodruff P. W. R., David A. S., Murray R. M., Bullmore E. T. (2000). Meta-analysis of regional brain volumes in schizophrenia. Am. J. Psychiatry 157, 16–25. 10.1176/ajp.157.1.16 PubMed DOI

Yoon U., Lee J.-M., Im K., Shin Y.-W., Cho B. H., Kim I. Y., et al. . (2007). Pattern classification using principal components of cortical thickness and its discriminative pattern in schizophrenia. Neuroimage 34, 1405–1415. 10.1016/j.neuroimage.2006.11.021 PubMed DOI

Zanetti M. V., Schaufelberger M. S., Doshi J., Ou Y., Ferreira L. K., Menezes P. R., et al. . (2013). Neuroanatomical pattern classification in a population-based sample of first-episode schizophrenia. Prog. Neuropsychopharmacol. Biol. Psychiatry 43, 116–125. 10.1016/j.pnpbp.2012.12.005 PubMed DOI

Structural MRI-Based Schizophrenia Classification Using Autoencoders and 3D Convolutional Neural Networks in Combination with Various Pre-Processing Techniques