INTRODUCTION: A rapid and reliable neuropsychological protocol is essential for the efficient assessment of neurocognitive constructs related to emergent neurodegenerative diseases. We developed an AI-assisted, digitally administered/scored neuropsychological protocol that can be remotely administered in ~10 min. This protocol assesses the requisite neurocognitive constructs associated with emergent neurodegenerative illnesses. METHODS: The protocol was administered to 77 ambulatory care/memory clinic patients (56.40% women; 88.50% Caucasian). The protocol includes a 6-word version of the Philadelphia (repeatable) Verbal Learning Test [P(r)VLT], three trials of 5 digits backward from the Backwards Digit Span Test (BDST), and the "animal" fluency test. The protocol provides a comprehensive set of traditional "core" measures that are typically obtained through paper-and-pencil tests (i.e., serial list learning, immediate and delayed free recall, recognition hits, percent correct serial order backward digit span, and "animal" fluency output). Additionally, the protocol includes variables that quantify errors and detail the processes used in administering the tests. It also features two separate, norm-referenced summary scores specifically designed to measure executive control and memory. RESULTS: Using four core measures, we used cluster analysis to classify participants into four groups: cognitively unimpaired (CU; n = 23), amnestic mild cognitive impairment (MCI; n = 17), dysexecutive MCI (n = 23), and dementia (n = 14). Subsequent analyses of error and process variables operationally defined key features of amnesia (i.e., rapid forgetting, extra-list intrusions, profligate responding to recognition foils); key features underlying reduced executive abilities (i.e., BDST items and dysexecutive errors); and the strength of the semantic association between successive responses on the "animal" fluency test. Executive and memory index scores effectively distinguished between all four groups. There was over 90% agreement between how cluster analysis of digitally obtained measures classified patients compared to classification using a traditional comprehensive neuropsychological protocol. The correlations between digitally obtained outcome variables and analogous paper/pencil measures were robust. DISCUSSION: The digitally administered protocol demonstrated a capacity to identify patterns of impaired performance and classification similar to those observed with standard paper/pencil neuropsychological tests. The inclusion of both core measures and detailed error/process variables suggests that this protocol can detect subtle, nuanced signs of early emergent neurodegenerative illness efficiently and comprehensively.

Columbia University Medical Center Department of Neurology Cognitive Neuroscience Division Taub Institute and Sergievsky Center New York NY United States

Department of Clinical and Health Psychology University of Florida Gainesville FL United States

Department of Geriatrics and Gerontology New Jersey Institute for Successful Aging Rowan Virtua School of Osteopathic Medicine Glassboro NJ United States

Department of Neurology and Center of Clinical Neuroscience 1st Faculty of Medicine Charles University Prague Czechia

Department of Neurology Harvard Medical School Boston MA United States

Department of Psychiatry and Behavioral Health University of North Dakota School of Medicine and Health Sciences Grand Forks ND United States

Department of Psychology Rowan University Glassboro NJ United States

Hinda and Arthur Marcus Institute for Aging Research and Deanna Sidney Wolk Center for Memory Health Hebrew Senior Life Boston MA United States

Linus Health Boston MA United States

Rush Alzheimer's Disease Center and the Department of Psychiatry and Behavioral Sciences Rush University Medical Center Chicago IL United States

See more in PubMed

Alzheimer's Association (2023). Alzheimer's disease facts and figures. Alzheimers Dement. 19, 1598–1695. 10.1002/alz.13016 PubMed DOI

Bauer R. M., Reckess G. Z., Kumar A., Valenstein E. (2012). “Amnestic disorders,” in Clinical Neuropsychology, 5th Edn, eds. K. M Heilman, and E. Valenstein (New Yor, NY: Oxford University Press; ), 504–581.

Binaco R., Calzaretto N., Epifano J., McGuire S., Umer M., Emrani S., et al. . (2020). Machine learning analysis of digital clock drawing test performance for differential classification of mild cognitive impairment subtypes versus Alzheimer's disease. J. Int. Neuropsychol. Soc. 26, 690–700. 10.1017/S1355617720000144 PubMed DOI

Bondi M. W., Monsch A. U., Galasko D., Butters N., Salmon D. P., Delis D. C. (1994). Preclinical cognitive markers of dementia of the Alzheimer type. Neuropsychology 8, 374–384 10.1037/0894-4105.8.3.374 DOI

Bradford A., Kunik M. E., Schulz P., Williams S. P., Singh H. (2009). Missed and delayed diagnosis of dementia in primary care: prevalence and contributing factors. Alzheimers Dis. Assoc. Disord. 23, 306–314. 10.1097/WAD.0b013e3181a6bebc PubMed DOI PMC

Butters N., Miliotis P. (1985). “Amnesic disorders,” in Clinical Neuropsychology, 2nd Edn, eds. K. M. Heilman, and E. Valenstein (New York, NY: Oxford University Press; ), 403–451.

Cosentino S., Jefferson A., Chute D. L., Kaplan E., Libon D. J. (2004). Clock drawing errors in dementia: neuropsychological and neuroanatomical considerations. Cogn. Behav. Neurol. 17, 74–84. 10.1097/01.wnn.0000119564.08162.46 PubMed DOI

Davoudi A., Dion C., Amini S., Tighe P. J., Price C. C., Libon D. J., et al. . (2021). Classifying non-dementia and Alzheimer's disease/vascular dementia patients using kinematic, time-based, and visuospatial parameters: The Digital Clock Drawing Test. J. Alzheimers Dis. 82, 47–57. 10.3233/JAD-201129 PubMed DOI PMC

Delis D. C., Kramer J. H., Kaplan E., Ober B. A. (1987). The California Verbal Learning Test, 1st Edn. San Antonio, TX: The Psychological Corporation.

Delis D. C., Massman P. J., Butters N., Salmon D. P., Kramer J. H., Cermak L. (1991). Profiles of demented and amnesic patients on the California Verbal Learning Test: implications for the assessment of memory disorders. Psychol. Assess. 3, 19–26. 10.1037/1040-3590.3.1.19 DOI

Dion C., Arias F., Amini S., Davis R., Penney D., Libon D. J., et al. . (2020). Cognitive correlates of Digital Clock Drawing metrics in older adults with and without mild cognitive impairment. J. Alzheimers Dis. 75, 73–83. 10.3233/JAD-191089 PubMed DOI PMC

Dion C., Tanner J. J., Formanski E. M., Davoudi A., Rodriguez K., Wiggins M. E., et al. . (2022). The functional connectivity and neuropsychology underlying mental planning operations: data from the digital clock drawing test. Front. Aging Neurosci. 14:868500. 10.3389/fnagi.2022.868500 PubMed DOI PMC

Emrani S., Lamar M., Price C., Baliga S., Wasserman V., Matusz E. F., et al. . (2021a). Neurocognitive constructs underlying executive control in statistically determined mild cognitive impairment. J. Alzheimers Dis. 82, 5–16. 10.3233/JAD-201125 PubMed DOI

Emrani S., Lamar M., Price C. C., Baliga S., Wasserman V., Matusz E., et al. . (2021b). Assessing the capacity for mental manipulation in patients with statistically determined mild cognitive impairment using digital technology. Explor. Med. 2, 86–97. 10.37349/emed.2021.00034 DOI

Emrani S., Lamar M., Price C. C., Swenson R., Libon D. J., Baliga G. (2023). Neurocognitive operations underlying working memory abilities: an analysis of latency and time-based parameters. J. Alzheimers Dis. 94, 1535–1547. 10.3233/JAD-230288 PubMed DOI PMC

Emrani S., Libon D. J., Lamar M., Price C. C., Jefferson A. L., Gifford K. A., et al. . (2018). Assessing working memory in mild cognitive impairment with serial order recall. J. Alzheimers Dis. 61, 917–928. 10.3233/JAD-170555 PubMed DOI PMC

Eppig J., Wambach D., Nieves C., Price C. C., Lamar M., Delano-Wood L., et al. . (2012). Dysexecutive functioning in mild cognitive impairment: derailment in temporal gradients. J. Int. Neuropsychol. Soc. 18, 20–28. 10.1017/S1355617711001238 PubMed DOI PMC

Folstein M. F., Folstein S. E., McHugh P. R. (1975). “Mini-mental state”. A practical method for grading the cognitive state of patients for the clinician. J. Psychiatr. Res. 12, 189–198. 10.1016/0022-3956(75)90026-6 PubMed DOI

Geraudie A., Battista P., García A. M., Allen I. E., Miller Z. A., Gorno-Tempini M. L., et al. . (2021). Speech and language impairments in behavioral variant frontotemporal dementia: a systematic review. Neurosci. Biobehav. Rev. 131, 1076–1095. 10.1016/j.neubiorev.2021.10.015 PubMed DOI

Geschwind N., Kaplan E. (1962). A human cerebral deconnection syndrome. A preliminary report. Neurology 12, 675–685. 10.1212/WNL.12.10.675 PubMed DOI

Giannouli V. (2023). Financial capacity assessments and AI: a Greek drama for geriatric psychiatry? Int. J. Geriatr. Psychiatry 38:e6008. 10.1002/gps.6008 PubMed DOI

Giovannetti T., Lamar M., Cloud B. S., Grossman M., Libon D. J. (1997). Impairment in category fluency in ischemic vascular dementia. Neuropsychology 11, 400–412. 10.1037/0894-4105.11.3.400 PubMed DOI

Gourovitch M. L., Kirkby B. S., Goldberg T. E., Weinberger D. R., Gold J. M., Esposito G., et al. . (2000). A comparison of rCBF patterns during letter and semantic fluency. Neuropsychology 14, 353–360. 10.1037/0894-4105.14.3.353 PubMed DOI

Heaton R. K., Miller S., Taylor M., Grant I. (2004). Revised Comprehensive Norms for an Expanded Halstead-Reitan Battery: Demographically Adjusted Neuropsychological Norms for African American and Caucasian Adults Scoring Programs. Lutz, FL: Psychological Assessment Resources.

Hurlstone M. J., Hitch G. J., Baddeley A. D. (2014). Memory for serial order across domains: an overview of the literature and directions for future research. Psychol. Bull. 140, 339–373. 10.1037/a0034221 PubMed DOI

Institute of Medicine (2001). Crossing the Quality Chasm: A New Health System for the 21st Century. Washington, DC: National Academies Press. PubMed

Kaplan E. (1988). “A process approach to neuropsychological assessment,” in Clinical Neuropsychology and Brain Function: Research, Measurement, and Practice, eds. T. Boll, and B. K. Bryant (Washington, DC: American Psychological Association).

Kaplan E. (1990). The process approach to neuropsychological assessment of psychiatric patients. J. Neuropsychiatry Clin. Neurosci. 2, 72–87. 10.1176/jnp.2.1.72 PubMed DOI

Kaplan E., Fein D., Morris R., Delis D. (1991). The WAIS-R as a Neuropsychological Instrument. San Antonio, TX: The Psychological Corporation.

Lamar M., Catani M., Price C. C., Heilman K. M., Libon D. J. (2008). The impact of region-specific leukoaraiosis on working memory deficits in dementia. Neuropsychologia 46, 2597–2601. 10.1016/j.neuropsychologia.2008.04.007 PubMed DOI

Lamar M., Price C. C., Giovannetti T., Swenson R., Libon D. J. (2010). The dysexecutive syndrome associated with ischaemic vascular disease and related subcortical neuropathology: a Boston process approach. Behav. Neurol. 22, 53–62. 10.1155/2010/505979 PubMed DOI PMC

Lamar M., Price C. C., Libon D. J., Penney D. L., Kaplan E., Grossman M., et al. . (2007). Alterations in working memory as a function of leukoaraiosis in dementia. Neuropsychologia 45, 245–254. 10.1016/j.neuropsychologia.2006.07.009 PubMed DOI PMC

Libon D. J., Penney D., Davis R., Tabby D., Eppig J., Nieves C., et al. . (2014). Deficits in processing speed and decision making in relapsing-remitting multiple sclerosis: The Digit Clock Drawing Test (dCDT). J. Multiple Scler. 1:113. 10.4172/jmso.1000113 DOI

Libon D. J., Bondi M. W., Price C. C., Lamar M., Eppig J., Wambach D. M., et al. . (2011). Verbal serial list learning in mild cognitive impairment: a profile analysis of interference, forgetting, and errors. J. Int. Neuropsychol. Soc. 17, 905–914. 10.1017/S1355617711000944 PubMed DOI PMC

Libon D. J., Lamar M., Price C. C., Jefferson A. L., Swenson R., Au R. (2018). “Neuropsychological evaluation for vascular dementia,” in APA Handbook of Dementia, eds. G. Smith, and S. T. Frias (Washington, DC: American Psychological Association).

Libon D. J., Malamut B. L., Swenson R., Sands L. P., Cloud B. S. (1996). Further analyses of clock drawings among demented and nondemented older subjects. Arch. Clin. Neuropsychol. 11, 193–205. 10.1093/arclin/11.3.193 PubMed DOI

Libon D. J., Matusz E. F., Cosentino S., Price C. C., Swenson R., Vermeulen M., et al. . (2023a). Using digital assessment technology to detect neuropsychological problems in primary care settings. Front. Psychol. 14:1280593. 10.3389/fpsyg.2023.1280593 PubMed DOI PMC

Libon D. J., McMillan C., Gunawardena D., Powers C., Massimo L., Khan A., et al. . (2009). Neurocognitive contributions to verbal fluency deficits in frontotemporal lobar degeneration. Neurology 73, 535–542. 10.1212/WNL.0b013e3181b2a4f5 PubMed DOI PMC

Libon D. J., Rascovsky K., Powers J., Irwin D. J., Boller A., Weinberg D., et al. . (2013). Comparative semantic profiles in semantic dementia and Alzheimer's disease. Brain 136 (Pt 8), 2497–2509. 10.1093/brain/awt165 PubMed DOI PMC

Libon D. J., Swenson R., Banks R., Schulman D., Higgins C., Pobst J., et al. . (2023b). Double dissociation of verbal serial list learning and semantic fluency test performance through automated analysis of acoustics. Front. Neurol.

Libon D. J., Swenson R., Lamar M., Price C. C., Baliga G., Pascual-Leone A., et al. . (2022). The Boston process approach and digital neuropsychological assessment: past research and future directions. J. Alzheimers Dis. 87, 1419–1432. 10.3233/JAD-220096 PubMed DOI

Libon D. J., Swenson R. A., Barnoski E. J., Sands L. P. (1993). Clock drawing as an assessment tool for dementia. Arch. Clin. Neuropsychol. 8, 405–415. 10.1093/arclin/8.5.405 PubMed DOI

Mahon E., Lachman M. E. (2022). Voice biomarkers as indicators of cognitive changes in middle and later adulthood. Neurobiol. Aging 119, 22–35. 10.1016/j.neurobiolaging.2022.06.010 PubMed DOI PMC

Mattke S., Batie D., Chodosh J., Felten K., Flaherty E., Fowler N. R., et al. . (2023). Expanding the use of brief cognitive assessments to detect suspected early-stage cognitive impairment in primary care. Alzheimers Dement. 19, 4252–4259. 10.1002/alz.13051 PubMed DOI

Matusz E. F., Price C. C., Lamar M., Swenson R., Au R., Emrani S., et al. . (2023). Dissociating statistically determined normal cognitive abilities and mild cognitive impairment subtypes with DCTclock. J. Int. Neuropsychol. Soc. 29, 148–158. 10.1017/S1355617722000091 PubMed DOI PMC

Mummery C. J., Patterson K., Hodges J. R., Wise R. J. (1996). Generating 'tiger' as an animal name or a word beginning with T: differences in brain activation. Proc. Biol. Sci. 263, 989–995. 10.1098/rspb.1996.0146 PubMed DOI

Nasreddine Z. S., Phillips N. A., Bédirian V., Charbonneau S., Whitehead V., Collin I., et al. . (2005). The Montreal Cognitive Assessment, MoCA: a brief screening tool for mild cognitive impairment. J. Am. Geriatr. Soc. 53, 695–699. 10.1111/j.1532-5415.2005.53221.x PubMed DOI

Penney D. L., Davis R., Libon D. J., Lamar M., Price C. C., Swenson R., et al. . (2011a). “The Digital Clock Drawing Test (dCDT) - II: A new computerized quantitative System,” in Abstract Presented at the 39TH Annual Meeting of the International Neuropsychological Society (Boston, MA: ).

Penney D. L., Libon D. J., Lamar M., Swenson R., Price C. C., Weninger C., et al. . (2011b). “The Digital Clock Drawing Test (dCDT) - III: Clinician reliability for a new quantitative system,” in Abstract Presented at the 39TH Annual Meeting of the International Neuropsychological Society (Boston, MA: ).

Penney D. L., Libon D. J., Price C. C., Lamar M., Swenson R., Garrett K., et al. . (2011c). “Digital Clock Drawing Test (dCBT) - IV: Total clock drawing and inter-stroke latencies or information revealed between the lines,” in Abstract Presented at the 39TH Annual Meeting of the International Neuropsychological Society (Boston, MA: ).

Piers R. J., Devlin K. N., Ning B., Liu Y., Wasserman B., Massaro J. M., et al. . (2017). Age and graphomotor decision making assessed with the digital clock drawing test: the Framingham Heart Study. J. Alzheimers Dis. 60, 1611–1620. 10.3233/JAD-170444 PubMed DOI PMC

Price C. C., Garrett K. D., Jefferson A. L., Cosentino S., Tanner J. J., Penney D. L., et al. . (2009). Leukoaraiosis severity and list-learning in dementia. Clin. Neuropsychol. 23, 944–961. 10.1080/13854040802681664 PubMed DOI PMC

Rascovsky K., Salmon D. P., Hansen L. A., Thal L. J., Galasko D. (2007). Disparate letter and semantic category fluency deficits in autopsy-confirmed frontotemporal dementia and Alzheimer's disease. Neuropsychology 21, 20–30. 10.1037/0894-4105.21.1.20 PubMed DOI

Sims J. R., Zimmer J. A., Evans C. D., Lu M., Ardayfio P., Sparks J., et al. . (2023). Donanemab in early symptomatic Alzheimer disease: the TRAILBLAZER-ALZ 2 randomized clinical trial. JAMA 330, 512–527. 10.1001/jama.2023.13239 PubMed DOI PMC

Souillard-Mandar W., Davis R., Rudin C., Au R., Libon D. J., Swenson R., et al. . (2016). Learning classification models of cognitive conditions from subtle behaviors in the digital clock drawing test. Mach. Learn. 102, 393–441. 10.1007/s10994-015-5529-5 PubMed DOI PMC

Souillard-Mandar W., Penney D., Schaible B., Pascual-Leone A., Au R., Davis R. (2021). DCTclock: clinically interpretable and automated artificial intelligence analysis of drawing behavior for capturing cognition. Fronti. Digit. Health 3:750661. 10.3389/fdgth.2021.750661 PubMed DOI PMC

Thomas K. R., Eppig J., Edmonds E. C., Jacobs D. M., Libon D. J., Au R., et al. . (2018). Word-list intrusion errors predict progression to mild cognitive impairment. Neuropsychology 32, 235–245. 10.1037/neu0000413 PubMed DOI PMC

van Dyck C. H., Swanson C. J., Aisen P., Bateman R. J., Chen C., Gee M., et al. . (2023). Lecanemab in early Alzheimer's disease. N. Engl. J. Med. 388, 9–21. 10.1056/NEJMoa2212948 PubMed DOI

Wechsler D. (1997). The Wechsler Adult Intelligence Scale-III. San Antonio, TX: Psychological Corporation.