INTRODUCTION: Dysarthria, a motor speech disorder caused by muscle weakness or paralysis, severely impacts speech intelligibility and quality of life. The condition is prevalent in motor speech disorders such as Parkinson's disease (PD), atypical parkinsonism such as progressive supranuclear palsy (PSP), Huntington's disease (HD), and amyotrophic lateral sclerosis (ALS). Improving intelligibility is not only an outcome that matters to patients but can also play a critical role as an endpoint in clinical research and drug development. This study validates a digital measure for speech intelligibility, the ki: SB-M intelligibility score, across various motor speech disorders and languages following the Digital Medicine Society (DiMe) V3 framework. METHODS: The study used four datasets: healthy controls (HCs) and patients with PD, HD, PSP, and ALS from Czech, Colombian, and German populations. Participants' speech intelligibility was assessed using the ki: SB-M intelligibility score, which is derived from automatic speech recognition (ASR) systems. Verification with inter-ASR reliability and temporal consistency, analytical validation with correlations to gold standard clinical dysarthria scores in each disease, and clinical validation with group comparisons between HCs and patients were performed. RESULTS: Verification showed good to excellent inter-rater reliability between ASR systems and fair to good consistency. Analytical validation revealed significant correlations between the SB-M intelligibility score and established clinical measures for speech impairments across all patient groups and languages. Clinical validation demonstrated significant differences in intelligibility scores between pathological groups and healthy controls, indicating the measure's discriminative capability. DISCUSSION: The ki: SB-M intelligibility score is a reliable, valid, and clinically relevant tool for assessing speech intelligibility in motor speech disorders. It holds promise for improving clinical trials through automated, objective, and scalable assessments. Future studies should explore its utility in monitoring disease progression and therapeutic efficacy as well as add data from further dysarthrias to the validation.

Centre de Mémoire de Ressources et de Recherche Centre Hospitalier Universitaire Nice Nice France

Cobtek Lab University Côte d'azur Nice France

Department of Circuit Theory Czech Technical University Prague Prague Czechia

Department of Neurology Faculty of Medicine and University Hospital Cologne Cologne Germany

GITA Lab Faculty of Engineering University of Antioquia Medellín Colombia

IfL Phonetics Faculty of Arts and Humanities University of Cologne Cologne Germany

ki elements GmbH Saarbrücken Germany

Pattern Recognition Lab Friedrich Alexander Universität Erlangen Nürnberg Erlangen Germany

Erratum v

Front Digit Health. 2024 Nov 13;6:1488178. doi: 10.3389/fdgth.2024.1488178. PubMed

Zobrazit více v PubMed

Darley FL, Aronson AE, Brown JR. Differential diagnostic patterns of dysarthria. J Speech Hear Res. (1969) 12(2):246–69. 10.1044/jshr.1202.246 PubMed DOI

Amato F, Borzì L, Olmo G, Artusi CA, Imbalzano G, Lopiano L. Speech impairment in Parkinson’s disease: acoustic analysis of unvoiced consonants in Italian native speakers. IEEE Access. (2021) 9:166370–81. 10.1109/ACCESS.2021.3135626 DOI

Rusz J, Cmejla R, Tykalova T, Ruzickova H, Klempir J, Majerova V, et al. Imprecise vowel articulation as a potential early marker of Parkinson’s disease: effect of speaking task. J Acoust Soc Am. (2013) 134(3):2171–81. 10.1121/1.4816541 PubMed DOI

Rusz J, Tykalová T, Salerno G, Bancone S, Scarpelli J, Pellecchia MT. Distinctive speech signature in cerebellar and parkinsonian subtypes of multiple system atrophy. J Neurol. (2019) 266(6):1394–404. 10.1007/s00415-019-09271-7 PubMed DOI

Rusz J, Tykalova T, Ramig LO, Tripoliti E. Guidelines for speech recording and acoustic analyses in dysarthrias of movement disorders. Mov Disord. (2021) 36(4):803–14. 10.1002/mds.28465 PubMed DOI

Daoudi K, Das B, Tykalova T, Klempir J, Rusz J. Speech acoustic indices for differential diagnosis between Parkinson’s disease, multiple system atrophy and progressive supranuclear palsy. NPJ Parkinsons Dis. (2022) 8(1):1–13. 10.1038/s41531-021-00272-w PubMed DOI PMC

Kim Y, Kent RD, Kent JF, Duffy JR. Perceptual and acoustic features of dysarthria in multiple system atrophy. J Med Speech-Lang Pathol. (2010) 18(4):66–71.

Diehl SK, Mefferd AS, Lin YC, Sellers J, McDonell KE, De Riesthal M, et al. Motor speech patterns in Huntington disease. Neurology. (2019) 93(22):E2042–52. 10.1212/WNL.0000000000008541 PubMed DOI PMC

Kouba T, Frank W, Tykalova T, Mühlbäck A, Klempíř J, Lindenberg KS, et al. Speech biomarkers in Huntington disease: a cross-sectional study in pre-symptomatic, prodromal and early manifest stages. Eur J Neurol. (2023) 1262–71. 10.1111/ene.15726 PubMed DOI

Tomik B, Guiloff RJ. Dysarthria in amyotrophic lateral sclerosis: a review. Amyotroph Lateral Scler. (2010) 11(1–2):4–15. 10.3109/17482960802379004 PubMed DOI

Darley FL, Brown JR, Goldstein NP. Dysarthria in multiple sclerosis. J Speech Hear Res. (1972) 15(2):229–45. 10.1044/jshr.1502.229 PubMed DOI

Rusz J, Benova B, Ruzickova H, Novotny M, Tykalova T, Hlavnicka J, et al. Characteristics of motor speech phenotypes in multiple sclerosis. Mult Scler Relat Disord. (2018) 19:62–9. 10.1016/j.msard.2017.11.007 PubMed DOI

Van Uem JMT, Marinus J, Canning C, Van Lummel R, Dodel R, Liepelt-Scarfone I, et al. Health-related quality of life in patients with Parkinson’s disease—a systematic review based on the ICF model. Neurosci Biobehav Rev. (2016) 61:26–34. 10.1016/j.neubiorev.2015.11.014 PubMed DOI

Chu SY, Tan CL. Subjective self-rated speech intelligibility and quality of life in patients with Parkinson’s disease in a Malaysian sample. Open Public Health J. (2018) 11(1):485–93. 10.2174/1874944501811010485 DOI

McAuliffe MJ, Baylor CR, Yorkston KM. Variables associated with communicative participation in Parkinson’s disease and its relationship to measures of health-related quality-of-life. Int J Speech Lang Pathol. (2017) 19(4):407–17. 10.1080/17549507.2016.1193900 PubMed DOI PMC

Schrag A, Jahanshahi M, Quinn N. How does Parkinson’s disease affect quality of life? A comparison with quality of life in the general population. Mov Disord. (2000) 15(6):1112–8. 10.1002/1531-8257(200011)15:6<1112::AID-MDS1008>3.0.CO;2-A PubMed DOI

Yorkston KM, Yorkston KM. Management of motor speech disorders in children and adults. 2nd ed. Austin, TX: Pro-Ed; (1999). p. 618.

Miller N, Allcock L, Jones D, Noble E, Hildreth AJ, Burn DJ. Prevalence and pattern of perceived intelligibility changes in Parkinson’s disease. J Neurol Neurosurg Amp Psychiatry. (2007) 78(11):1188–90. 10.1136/jnnp.2006.110171 PubMed DOI PMC

Gutz SE, Stipancic KL, Yunusova Y, Berry JD, Green JR. Validity of off-the-shelf automatic speech recognition for assessing speech intelligibility and speech severity in speakers with amyotrophic lateral sclerosis. J Speech Lang Hear Res. (2022) 65(6):2128–43. 10.1044/2022_JSLHR-21-00589 PubMed DOI PMC

Kent RD, Weismer G, Kent JF, Rosenbek JC. Toward phonetic intelligibility testing in dysarthria. J Speech Hear Disord. (1989) 54(4):482–99. 10.1044/jshd.5404.482 PubMed DOI

Goetz CG, Tilley BC, Shaftman SR, Stebbins GT, Fahn S, Martinez-Martin P, et al. Movement disorder society-sponsored revision of the unified Parkinson’s disease rating scale (MDS-UPDRS): scale presentation and clinimetric testing results: MDS-UPDRS: clinimetric assessment. Mov Disord. (2008) 23(15):2129–70. 10.1002/mds.22340 PubMed DOI

Cedarbaum JM, Stambler N, Malta E, Fuller C, Hilt D, Thurmond B, et al. The ALSFRS-R: a revised ALS functional rating scale that incorporates assessments of respiratory function. J Neurol Sci. (1999) 169(1–2):13–21. 10.1016/S0022-510X(99)00210-5 PubMed DOI

Fagherazzi G, Fischer A, Ismael M, Despotovic V. Voice for health: the use of vocal biomarkers from research to clinical practice. Digit Biomark. (2021) 5(1):78–88. 10.1159/000515346 PubMed DOI PMC

Espay AJ, Hausdorff JM, Sánchez-Ferro Á, Klucken J, Merola A, Bonato P, et al. A roadmap for implementation of patient-centered digital outcome measures in Parkinson’s disease obtained using mobile health technologies. Mov Disord. (2019) 34(5):657–63. 10.1002/mds.27671 PubMed DOI PMC

Salmon K, Genge A. “Clinical trials in ALS—current challenges and strategies for future directions”. In: Shaw CA, Morrice JR, editors. Spectrums of Amyotrophic Lateral Sclerosis. 1st ed. Hoboken, NJ: John Wiley & Sons, Inc. (2021). p. 161–80.

Bowden M, Beswick E, Tam J, Perry D, Smith A, Newton J, et al. A systematic review and narrative analysis of digital speech biomarkers in motor neuron disease. NPJ Digit Med. (2023) 6(1):228. 10.1038/s41746-023-00959-9 PubMed DOI PMC

Dorsey ER, Papapetropoulos S, Xiong M, Kieburtz K. The first frontier: digital biomarkers for neurodegenerative disorders. Digit Biomark. (2017) 1(1):6–13. 10.1159/000477383 PubMed DOI PMC

Vásquez-Correa JC, Orozco-Arroyave JR, Bocklet T, Nöth E. Towards an automatic evaluation of the dysarthria level of patients with Parkinson’s disease. J Commun Disord. (2018) 76:21–36. 10.1016/j.jcomdis.2018.08.002 PubMed DOI

De Russis L, Corno F. On the impact of dysarthric speech on contemporary ASR cloud platforms. J Reliab Intell Environ. (2019) 5(3):163–72. 10.1007/s40860-019-00085-y DOI

Mustafa MB, Rosdi F, Salim SS, Mughal MU. Exploring the influence of general and specific factors on the recognition accuracy of an ASR system for dysarthric speaker. Expert Syst Appl. (2015) 42(8):3924–32. 10.1016/j.eswa.2015.01.033 DOI

Keshet J. Automatic speech recognition: a primer for speech-language pathology researchers. Int J Speech Lang Pathol. (2018) 20(6):599–609. 10.1080/17549507.2018.1510033 PubMed DOI

Goldsack JC, Coravos A, Bakker JP, Bent B, Dowling AV, Fitzer-Attas C, et al. Verification, analytical validation, and clinical validation (V3): the foundation of determining fit-for-purpose for biometric monitoring technologies (BioMeTs). NPJ Digit Med. (2020) 3(1):1–15. 10.1038/s41746-020-0260-4 PubMed DOI PMC

Coravos A, Doerr M, Goldsack J, Manta C, Shervey M, Woods B, et al. Modernizing and designing evaluation frameworks for connected sensor technologies in medicine. NPJ Digit Med. (2020) 3(1):37. 10.1038/s41746-020-0237-3 PubMed DOI PMC

Goldsack JC, Dowling AV, Samuelson D, Patrick-Lake B, Clay I. Evaluation, acceptance, and qualification of digital measures: from proof of concept to endpoint. Digit Biomark. (2021) 5(1):53–64. 10.1159/000514730 PubMed DOI PMC

Rusz J, Klempíř J, Tykalová T, Baborová E, Čmejla R, Růžička E, et al. Characteristics and occurrence of speech impairment in Huntington’s disease: possible influence of antipsychotic medication. J Neural Transm. (2014) 121(12):1529–39. 10.1007/s00702-014-1229-8 PubMed DOI

Hlavnička J, Čmejla R, Tykalová T, Šonka K, Růžička E, Rusz J. Automated analysis of connected speech reveals early biomarkers of Parkinson’s disease in patients with rapid eye movement sleep behaviour disorder. Sci Rep. (2017) 7(1):12. 10.1038/s41598-017-00047-5 PubMed DOI PMC

Novotny M, Melechovsky J, Rozenstoks K, Tykalova T, Kryze P, Kanok M, et al. Comparison of automated acoustic methods for oral diadochokinesis assessment in amyotrophic lateral sclerosis. J Speech Lang Hear Res. (2020) 63(10):3453–60. 10.1044/2020_JSLHR-20-00109 PubMed DOI

Orozco-Arroyave JR, Arias-Londoño JD, Vargas-Bonilla JF, González-Rátiva MC, Nöth E. New Spanish speech corpus database for the analysis of people suffering from Parkinson’s disease. In: Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC’14); 26–31 May 2014; Reykjavik, Iceland. Paris: European Language Resources Association (ELRA) (2014). p. 342–7.

Thies T, Mücke D, Lowit A, Kalbe E, Steffen J, Barbe MT. Prominence marking in parkinsonian speech and its correlation with motor performance and cognitive abilities. Neuropsychologia. (2020) 137:107306. 10.1016/j.neuropsychologia.2019.107306 PubMed DOI

Hlavnička J. The dysarthria analyzer. Available online at: https://www.dysan.cz/ (accessed June 25, 2024).

Kieburtz K, Penney JB, Como P, Ranen N, Shoulson I, Feigin A, et al. Unified Huntington’s disease rating scale: reliability and consistency. Mov Disord. (1996) 11(2):136–42. 10.1002/mds.870110204 PubMed DOI

Payan CAM, Viallet F, Landwehrmeyer BG, Bonnet AM, Borg M, Durif F, et al. Disease severity and progression in progressive supranuclear palsy and multiple system atrophy: validation of the NNIPPS—Parkinson plus scale. PLoS One. (2011) 6(8):e22293. 10.1371/journal.pone.0022293 PubMed DOI PMC

Google LLC. Google speech-to-text V2.25.1 (2023). Available online at: https://cloud.google.com/speech-to-text/ (Accessed July 29, 2022).

Amazon Web Services Inc. Amazon Transcribe V1.20.15 (2023). Available online at: https://aws.amazon.com/pm/transcribe/ (Accessed April 11, 2024).

Van Nuffelen G, Middag C, De Bodt M, Martens J. Speech technology-based assessment of phoneme intelligibility in dysarthria. Int J Lang Commun Disord. (2009) 44(5):716–30. 10.1080/13682820802342062 PubMed DOI

Ratitch B, Trigg A, Majumder M, Vlajnic V, Rethemeier N, Nkulikiyinka R. Clinical validation of novel digital measures: statistical methods for reliability evaluation. Digit Biomark. (2023) 7:74–91. 10.1159/000531054 PubMed DOI PMC

Dimauro G, Di Nicola V, Bevilacqua V, Caivano D, Girardi F. Assessment of speech intelligibility in Parkinson’s disease using a speech-to-text system. IEEE Access. (2017) 5:22199–208. 10.1109/ACCESS.2017.2762475 DOI

Zhang Y, Han W, Qin J, Wang Y, Bapna A, Chen Z, et al. Google USM: scaling automatic speech recognition beyond 100 languages. arXiv [preprint]. (2023). Available online at: https://arxiv.org/abs/2303.01037 (accessed July 1, 2024).

Kim J, Kumar N, Tsiartas A, Li M, Narayanan SS. Automatic intelligibility classification of sentence-level pathological speech. Comput Speech Lang. (2015) 29(1):132–44. 10.1016/j.csl.2014.02.001 PubMed DOI PMC

Huang A, Hall K, Watson C, Shahamiri SR. A review of automated intelligibility assessment for dysarthric speakers. In: 2021 International Conference on Speech Technology and Human-Computer Dialogue (SpeD); 13–15 Oct 2021; Bucharest, Romania. New York, NY: IEEE; (2021). p. 19–24.

Stipancic KL, Tjaden K, Wilding G. Comparison of intelligibility measures for adults with Parkinson’s disease, adults with multiple sclerosis, and healthy controls. J Speech Lang Hear Res. (2016) 59(2):230–8. 10.1044/2015_JSLHR-S-15-0271 PubMed DOI PMC

Campi M, Peters GW, Toczydlowska D. Ataxic speech disorders and Parkinson’s disease diagnostics via stochastic embedding of empirical mode decomposition. PLoS One. (2023) 18(4):e0284667. 10.1371/journal.pone.0284667 PubMed DOI PMC

Rowe HP, Gutz SE, Maffei MF, Tomanek K, Green JR. Characterizing dysarthria diversity for automatic speech recognition: a tutorial from the clinical perspective. Front Comput Sci. (2022) 4:770210. 10.3389/fcomp.2022.770210 PubMed DOI PMC

Tu M, Wisler A, Berisha V, Liss JM. The relationship between perceptual disturbances in dysarthric speech and automatic speech recognition performance. J Acoust Soc Am. (2016) 140(5):EL416–22. 10.1121/1.4967208 PubMed DOI PMC

Arias-Vergara T, Vásquez-Correa JC, Orozco-Arroyave JR, Nöth E. Speaker models for monitoring Parkinson’s disease progression considering different communication channels and acoustic conditions. Speech Commun. (2018) 101:11–25. 10.1016/j.specom.2018.05.007 DOI