Over the past decade, neuropsychiatric fluctuations in Parkinson's disease (PD) have been increasingly recognized for their impact on patients' quality of life. Speech, a complex function carrying motor, emotional, and cognitive information, offers potential insights into these fluctuations. While previous studies have focused on acoustic analysis to assess motor speech disorders reliably, the potential of linguistic patterns associated with neuropsychiatric fluctuations in PD remains unexplored. This study analyzed the content of spontaneous speech from 33 PD patients in ON and OFF medication states, using machine learning and large language models (LLMs) to predict medication states and a neuropsychiatric state score. The top-performing model, the LLM Gemma-2 (9B), achieved 98% accuracy in differentiating ON and OFF states and its predicted scores were highly correlated with actual scores (Spearman's ρ = 0.81). These methods could provide a more comprehensive assessment of PD treatment effects, allowing remote neuropsychiatric symptom monitoring via mobile devices.

ARTORG Center for Biomedical Engineering Research AIMI University of Bern Bern Switzerland

ARTORG Center for Biomedical Engineering Research Gerontechnology and Rehabilitation Group University of Bern Bern Switzerland

Center for Networked Biomedical Research in Neurodegenerative Diseases Carlos 3 Institute of Health Madrid Spain

Department of Circuit Theory Faculty of Electrical Engineering Czech Technical University Prague Prague Czech Republic

Department of Neurology Bern University Hospital and University of Bern Bern Switzerland

Zobrazit více v PubMed

Bloem, B. R., Okun, M. S. & Klein, C. Parkinson’s disease. The Lancet397, 2284–2303 (2021). PubMed

Weintraub, D., Matthew, M. D. & Stern, B. Psychiatric Complications in Parkinson Disease. Am. J. Geriatr. Psychiatry13, 844–851 (2005). PubMed

Santos García, D. et al. Non-motor symptoms burden, mood, and gait problems are the most significant factors contributing to a poor quality of life in non-demented Parkinson’s disease patients: results from the COPPADIS Study Cohort. Parkinsonism Relat. Disord.66, 151–157 (2019). PubMed

Martínez-Fernández, R., Schmitt, E., Martinez-Martin, P. & Krack, P. The hidden sister of motor fluctuations in Parkinson’s disease: A review on nonmotor fluctuations. Mov. Disord.31, 1080–1094 (2016). PubMed

Castrioto, A., Lhommée, E., Moro, E. & Krack, P. Review mood and behavioural eff ects of subthalamic stimulation in Parkinson’s disease. Lancet Neurol.13, 287–305 (2014). PubMed

Pagonabarraga, J., Kulisevsky, J., Strafella, A. P. & Krack, P. Apathy in Parkinson’s disease: clinical features, neural substrates, diagnosis, and treatment. Lancet Neurol.14, 518–531 (2015). PubMed

Amstutz, D. et al. Management of impulse control disorders with subthalamic nucleus deep brain stimulation in Parkinson’s Disease. CNS Neurol. Disord. Drug Targets19, 611–617 (2020). PubMed

Debove, I. et al. Management of impulse control and related disorders in Parkinson’s Disease: an expert consensus. Mov. Disord. 39, 235–248 (2024). PubMed

Connolly, B. S. & Lang, A. E. Pharmacological treatment of Parkinson disease: a review. JAMA311, 1670–1683 (2014). PubMed

Rusz, J., Krack, P. & Tripoliti, E. From prodromal stages to clinical trials: the promise of digital speech biomarkers in Parkinson’s disease. Neurosci. Biobehav. Rev.167, 105922 (2024). PubMed

Rusz, J. et al. Quantitative assessment of motor speech abnormalities in idiopathic rapid eye movement sleep behaviour disorder. Sleep. Med.19, 141–147 (2016). PubMed

Rusz, J., Tykalová, T., Novotný, M., Růžička, E. & Dušek, P. Distinct patterns of speech disorder in early-onset and late-onset de-novo Parkinson’s disease. NPJ Parkinsons Dis.7, 98 (2021). PubMed PMC

Šubert, M. et al. Spoken language alterations can predict phenoconversion in isolated rapid eye movement sleep behavior disorder: a multicenter study. Ann. Neurol.95, 530–543 (2024). PubMed

Rusz, J. et al. Speech biomarkers in rapid eye movement sleep behavior disorder and Parkinson Disease. Ann. Neurol.90, 62–75 (2021). PubMed PMC

Pell, M. D. & Leonard, C. L. Processing emotional tone from speech in Parkinson’s disease: a role for the basal ganglia. Cognit. Affect Behav. Neurosci.10.3758/CABN.3.4.275 (2003). PubMed

Lausen, A. & Hammerschmidt, K. Emotion recognition and confidence ratings predicted by vocal stimulus type and prosodic parameters. Humanit Soc. Sci. Commun.7, 1–17 (2020).

Cao, H., Beňuš, Š., Gur, R. C., Verma, R. & Nenkova, A. Prosodic cues for emotion: analysis with discrete characterization of intonation. in Proc. International Conference on Speech Prosody 130–134 10.21437/speechprosody.2014-14. (International Speech Communications Association, 2014). PubMed PMC

Wan, T. M., Gunawan, T. S., Qadri, S. A. A., Kartiwi, M. & Ambikairajah, E. A comprehensive review of speech emotion recognition systems. IEEE Access10.1109/ACCESS.2021.3068045 (2021).

Sechidis, K., Fusaroli, R., Orozco-Arroyave, J. R., Wolf, D. & Zhang, Y. P. A machine learning perspective on the emotional content of Parkinsonian speech. Artif. Intell. Med.115, 102061 (2021). PubMed

Palmirotta, C. et al. Unveiling the Diagnostic Potential of Linguistic Markers in Identifying Individuals with Parkinson’s Disease through Artificial Intelligence: A Systematic Review. Brain Sci.10.3390/brainsci14020137 (2024). PubMed PMC

García, A. M. et al. How language flows when movements don’t: an automated analysis of spontaneous discourse in Parkinson’s disease. Brain Lang.162, 19–28 (2016). PubMed

Yokoi, K. et al. Analysis of spontaneous speech in Parkinson’s disease by natural language processing. Parkinsonism Relat. Disord.113, 105411 (2023). PubMed

Ash, S. et al. Longitudinal decline in speech production in Parkinson’s disease spectrum disorders. Brain Lang.171, 42–51 (2017). PubMed PMC

Šubert, M. et al. Linguistic abnormalities in isolated rapid eye movement sleep behavior disorder. Mov. Disord.37, 1872–1882 (2022). PubMed

Cevik, F. & Kilimci, Z. H. Analysis of Parkinson’s Disease using Deep Learning and Word Embedding Models. Acad. Perspect. Procedia2, 786–797 (2019).

Marras, C. et al. What Patients Say: Large-Scale Analyses of Replies to the Parkinson’s Disease Patient Report of Problems (PD-PROP). J. Parkinsons Dis.13, 757–767 (2023). PubMed PMC

Agarwal, C., Tanneru, S. H. & Lakkaraju, H. Faithfulness vs. plausibility: on the (Un)Reliability of explanations from large language models. arXiv preprint arXiv:2402.04614 (2024).

Huang, S., Mamidanna, S., Jangam, S., Zhou, Y. & Gilpin, L. H. Can large language models explain themselves? A study of LLM-generated self-explanations. arXiv preprint arXiv:2310.11207 (2023).

Xie, J., Chen, A. S., Lee, Y., Mitchell, E. & Finn, C. Calibrating Language Models with Adaptive Temperature Scaling. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pp. 18128–18138 (2024).

Schmitt, E. et al. Fluctuations in Parkinson’s disease and personalized medicine: bridging the gap with the neuropsychiatric fluctuation scale. Front. Neurol.14, 1242484 (2023). PubMed PMC

Norel, R. et al. Speech-based characterization of dopamine replacement therapy in people with Parkinson’s disease. NPJ Parkinsons Dis.6, 12 (2020). PubMed PMC

Zebaze, A., Sagot, B. & Bawden, R. In-Context Example Selection via Similarity Search Improves Low-Resource Machine Translation. arXiv preprint arXiv:2408.00397 (2024).

Portillo Wightman, G., DeLucia, A. & Dredze, M. Strength in Numbers: Estimating Confidence of Large Language Models by Prompt Agreement. In Proceedings of the 3rd Workshop on Trustworthy Natural Language Processing (TrustNLP 2023), pp. 326–362, Toronto, Canada. https://github.com/JHU-CLSP/ (2023).

Yang, K. et al. Towards Interpretable Mental Health Analysis with Large Language Models. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pp. 6056–6077 (2023).

Magalhães, A. D. et al. Subthalamic stimulation has acute psychotropic effects and improves neuropsychiatric fluctuations in Parkinson’s disease. BMJ Neurol. Open6, e000524 (2024). PubMed PMC

Dolev, E. L., Lutz, C. F. & Aepli, N. Does Whisper Understand Swiss German? An Automatic, Qualitative and Human Evaluation. in Proc. Eleventh Workshop on NLP for Similar Languages, Varieties, and Dialects (VarDial 2024) 28–40 (Association for Computational Linguistics, 2024).

Sicard, C., Zürich, E., Gillioz, V. & Pyszkowski, K. Spaiche: Extending State-of-the-Art ASR Models to Swiss German Dialects. in Proceedings of the 8th edition of the Swiss Text Analytics Conference 76–83 (Association for Computational Linguistics, 2023).

Koenecke, A., Choi, A. S. G., Mei, K. X., Schellmann, H. & Sloane, M. Careless Whisper: Speech-to-Text Hallucination Harms. in 2024 ACM Conference on Fairness, Accountability, and Transparency, FAccT 2024 1672–1681 10.1145/3630106.3658996. (Association for Computing Machinery, Inc, 2024).

Liu, C., Zhang, W., Zhao, Y., Luu, A. T. & Bing, L. Is translation all you need? A study on solving multilingual tasks with large language models. arXiv preprint arXiv:2403.10258 (2024).

Jin, R. et al. A Comprehensive Evaluation of Quantization Strategies for Large Language Models. In Findings of the Association for Computational Linguistics: ACL 2024, pp. 12186–12215 (2024).

Han, S., Mao, H. & Dally, W. J. Deep compression: compressing deep neural networks with pruning, trained quantization and huffman coding. In International Conference on Learning Representations (2016).

Illner, V. et al. Smartphone voice calls provide early biomarkers of Parkinsonism in rapid eye movement sleep behavior disorder. Mov. Disord.10.1002/mds.29921 (2024). PubMed

Delpont, B. et al. Psychostimulant effect of dopaminergic treatment and addictions in Parkinson’s disease. Mov. Disord.32, 1566–1573 (2017). PubMed

Postuma, R. B. et al. MDS clinical diagnostic criteria for Parkinson’s disease. Mov. Disord.30, 1591–1601 10.1002/mds.26424 (2015). PubMed

American Psychiatric Association. Diagnostic and Statistical Manual of Mental Disorders (DSM-5). American Psychiatric Pub, 5th edition. 10.1176/appi.books.9780890425596 (2013).

Schade, S., Mollenhauer, B. & Trenkwalder, C. Levodopa Equivalent Dose Conversion Factors: An Updated Proposal Including Opicapone and Safinamide. Mov. Disord. Clin. Pract. 7, 343–345 10.1002/mdc3.12921 (2020). PubMed PMC

Schmitt, E. et al. The neuropsychiatric fluctuations scale for Parkinson’s Disease: a pilot study. Mov. Disord. Clin. Pract.5, 265–272 (2018). PubMed PMC

Goetz, C. G. et al. Movement disorder society-sponsored revision of the unified Parkinson’s Disease Rating Scale (MDS-UPDRS): scale presentation and clinimetric testing results. Mov. Disord.23, 2129–2170 (2008). PubMed

Nasreddine, Z. S. et al. The montreal cognitive assessment, MoCA: a brief screening tool for mild cognitive impairment. J. Am. Geriatr. Soc.53, 695–699 (2005). PubMed

Zigmond, A. S. & Snaith, R. P. The hospital anxiety and depression scale. Acta Psychiatr. Scand.67, 361–370 (1983). PubMed

Starkstein, S. E. et al. Reliability, validity, and clinical correlates of apathy in Parkinson’s disease. J. Neuropsychiatry Clin. Neurosci.4, 134–139 (1992). PubMed

Weintraub, D. et al. Questionnaire for impulsive-compulsive disorders in Parkinson’s Disease–Rating Scale. Mov. Disord.27, 242–247 (2012). PubMed PMC

Rusz, J., Tykalova, T., Ramig, L. O. & Tripoliti, E. Guidelines for speech recording and acoustic analyses in dysarthrias of movement disorders. Mov. Disord. 36, 803–814. 10.1002/mds.28465 (2021). PubMed

Radford, A. et al. Robust speech recognition via large-scale weak supervision. in Proc. 40th International Conference on Machine Learning (JMLR.org, 2023).

Kuhn, K., Kersken, V., Reuter, B., Egger, N. & Zimmermann, G. Measuring the accuracy of automatic speech recognition solutions. ACM Trans. Access. Comput.16, 1–23 (2024).

Plüss, M. et al. STT4SG-350: a speech corpus for all Swiss German Dialect Regions. in Proc. 61st Annual Meeting of the Association for Computational Linguistics(Volume 2: Short Papers) vol. 2 1763–1772 (Short Papers, 2023).

Etxaniz, J., Azkune, G., Soroa, A., de Lacalle, O. L. & Artetxe, M. Do multilingual language models think better in English? In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (vol 2: Short Papers), pp. 550–564. Association for Computational Linguistics (2024).

Pedregosa, F. et al. Scikit-Learn: Machine Learning in Python.J. Mach. Learn. Res. 12, http://scikit-learn.sourceforge.net (2011).

Reimers, N. & Gurevych, I. Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pp. 3982–3992. Association for Computational Linguistics (2019).

Agbavor, F. & Liang, H. Predicting dementia from spontaneous speech using large language models. PLOS Digital Health1, e0000168 (2022). PubMed PMC

Vu, H., Abdurahman, S., Bhatia, S. & Ungar, L. Predicting Responses to Psychological Questionnaires from Participants’ Social Media Posts and Question Text Embeddings. In Findings of the Association for Computational Linguistics: EMNLP 2020. 10.18653/v1/2020.findings-emnlp.137 (2020).

Mathur, Y. et al. SummQA at MEDIQA-Chat 2023: In-Context Learning with GPT-4 for Medical Summarization. In Proceedings of the 5th Clinical Natural Language Processing Workshop, pp. 490–502, Association for Computational Linguistics (2023).

Naderalvojoud, B. & Hernandez-Boussard, T. Improving machine learning with ensemble learning on observational healthcare data. AMIA Annual Symposium Proceedings, 521–529 (2024). PubMed PMC

Vacareanu, R., Negru, V.-A., Suciu, V. & Surdeanu, M. From words to numbers: your large language model is secretly a capable regressor when given in-context examples. arXiv preprint arXiv:2404.07544 (2024).

Jiang, Z., Araki, J., Ding, H. & Neubig, G. How Can We Know When Language Models Know? On the Calibration of Language Models for Question Answering. Trans. Assoc. Comput. Linguist 962–977 10.1162/tacl (2021).

Pereira, T., Cardoso, S., Guerreiro, M., Mendonça, A. & Madeira, S. C. Targeting the uncertainty of predictions at patient-level using an ensemble of classifiers coupled with calibration methods, Venn-ABERS, and Conformal Predictors: A case study in AD. J. Biomed. Inf.101, 103350 (2020). PubMed

Guo, C., Pleiss, G., Sun, Y. & Weinberger, K. Q. On calibration of modern neural networks. in Proc.Machine Learning Research 1321–1330 (PMLR, 2017).