BACKGROUND: Despite the increasing use of remote measurement technologies (RMT) such as wearables or biosensors in health care programs, challenges associated with selecting and implementing these technologies persist. Many health care programs that use RMT rely on commercially available, "off-the-shelf" devices to collect patient data. However, validation of these devices is sparse, the technology landscape is constantly changing, relative benefits between device options are often unclear, and research on patient and health care provider preferences is often lacking. OBJECTIVE: To address these common challenges, we propose a novel device selection framework extrapolated from human-centered design principles, which are commonly used in de novo digital health product design. We then present a case study in which we used the framework to identify, test, select, and implement off-the-shelf devices for the Remote Assessment of Disease and Relapse-Central Nervous System (RADAR-CNS) consortium, a research program using RMT to study central nervous system disease progression. METHODS: The RADAR-CNS device selection framework describes a human-centered approach to device selection for mobile health programs. The framework guides study designers through stakeholder engagement, technology landscaping, rapid proof of concept testing, and creative problem solving to develop device selection criteria and a robust implementation strategy. It also describes a method for considering compromises when tensions between stakeholder needs occur. RESULTS: The framework successfully guided device selection for the RADAR-CNS study on relapse in multiple sclerosis. In the initial stage, we engaged a multidisciplinary team of patients, health care professionals, researchers, and technologists to identify our primary device-related goals. We desired regular home-based measurements of gait, balance, fatigue, heart rate, and sleep over the course of the study. However, devices and measurement methods had to be user friendly, secure, and able to produce high quality data. In the second stage, we iteratively refined our strategy and selected devices based on technological and regulatory constraints, user feedback, and research goals. At several points, we used this method to devise compromises that addressed conflicting stakeholder needs. We then implemented a feedback mechanism into the study to gather lessons about devices to improve future versions of the RADAR-CNS program. CONCLUSIONS: The RADAR device selection framework provides a structured yet flexible approach to device selection for health care programs and can be used to systematically approach complex decisions that require teams to consider patient experiences alongside scientific priorities and logistical, technical, or regulatory constraints.

Department for Neurosciences Center for Contextual Psychiatry Katholieke Universiteit Leuven Leuven Belgium

Department of Anthropology and Human Genetics Faculty of Science Charles University Prague Czech Republic

Department of Engineering and Applied Science University of Bergamo Bergamo Italy

Epidemiology Biostatistics and Prevention Institute University of Zürich Zürich Switzerland

Institute of Psychiatry Psychology and Neuroscience King's College London London United Kingdom

Janssen Pharmaceutica NV Beerse Belgium

Merck Research Labs Information Technology Merck Sharpe and Dohme Prague Czech Republic

Merck Research Labs Information Technology Merck Sharpe and Dohme Singapore Singapore

National Institute for Health Research Maudsley Biomedical Research Centre South London and Maudsley NHS Foundation Trust London United Kingdom

Neurology Services San Raffaele Hospital Multiple Sclerosis Centre Milan Italy

Patient Advisory Board Remote Assessment of Disease and Relapse Research Program King's College London London United Kingdom

Pharmacokinetics Pharmacodynamics and Drug Metabolism Merck and Co Inc Kenilworth NJ United States

Research and Development Information Technology Janssen Research and Development LLC Titusville NJ United States

Zobrazit více v PubMed

Noah B, Keller MS, Mosadeghi S, Stein L, Johl S, Delshad S, Tashjian VC, Lew D, Kwan JT, Jusufagic A, Spiegel BMR. Impact of remote patient monitoring on clinical outcomes: an updated meta-analysis of randomized controlled trials. NPJ Digit Med. 2018;1:20172. doi: 10.1038/s41746-017-0002-4. PubMed DOI PMC

Dorsey ER, Topol EJ. State of Telehealth. N Engl J Med. 2016 Jul 14;375(2):154–61. doi: 10.1056/NEJMra1601705. PubMed DOI

Marziniak M, Brichetto G, Feys P, Meyding-Lamadé U, Vernon K, Meuth SG. The use of digital and remote communication technologies as a tool for multiple sclerosis management: narrative review. JMIR Rehabil Assist Technol. 2018 Apr 24;5(1):e5. doi: 10.2196/rehab.7805. PubMed DOI PMC

Polhemus AM, Kadhim H, Barnes S, Zebrowski SE, Simmonds A, Masand SN, Banner J, Dupont M. Accelerating adoption of patient-facing technologies in clinical trials: a pharmaceutical industry perspective on opportunities and challenges. Ther Innov Regul Sci. 2019 Jan;53(1):8–24. doi: 10.1177/2168479018801566. PubMed DOI

Davis MM, Freeman M, Kaye J, Vuckovic N, Buckley DI. A systematic review of clinician and staff views on the acceptability of incorporating remote monitoring technology into primary care. Telemed J E Health. 2014 May;20(5):428–38. doi: 10.1089/tmj.2013.0166. PubMed DOI PMC

Perry B, Herrington W, Goldsack JC, Grandinetti CA, Vasisht KP, Landray MJ, Bataille L, DiCicco RA, Bradley C, Narayan A, Papadopoulos EJ, Sheth N, Skodacek K, Stem K, Strong TV, Walton MK, Corneli A. Use of mobile devices to measure outcomes in clinical research, 2010-2016: a systematic literature review. Digit Biomark. 2018;2(1):11–30. doi: 10.1159/000486347. PubMed DOI PMC

Vegesna A, Tran M, Angelaccio M, Arcona S. Remote patient monitoring via non-invasive digital technologies: a systematic review. Telemed J E Health. 2017 Jan;23(1):3–17. doi: 10.1089/tmj.2016.0051. PubMed DOI PMC

Marcolino MS, Oliveira JAQ, D'Agostino M, Ribeiro AL, Alkmim MBM, Novillo-Ortiz D. The impact of mHealth interventions: systematic review of systematic reviews. JMIR Mhealth Uhealth. 2018 Jan 17;6(1):e23. doi: 10.2196/mhealth.8873. PubMed DOI PMC

Lobelo F, Kelli HM, Tejedor SC, Pratt M, McConnell MV, Martin SS, Welk GJ. The wild wild west: a framework to integrate mHealth software applications and wearables to support physical activity assessment, counseling and interventions for cardiovascular disease risk reduction. Prog Cardiovasc Dis. 2016;58(6):584–94. doi: 10.1016/j.pcad.2016.02.007. PubMed DOI PMC

Wilson K, Bell C, Wilson L, Witteman H. Agile research to complement agile development: a proposal for an mHealth research lifecycle. NPJ Digit Med. 2018;1:46. doi: 10.1038/s41746-018-0053-1. PubMed DOI PMC

Chu AHY, Ng SHX, Paknezhad M, Gauterin A, Koh D, Brown MS, Müller-Riemenschneider F. Comparison of wrist-worn Fitbit Flex and waist-worn ActiGraph for measuring steps in free-living adults. PLoS One. 2017;12(2):e0172535. doi: 10.1371/journal.pone.0172535. PubMed DOI PMC

Dooley EE, Golaszewski NM, Bartholomew JB. Estimating accuracy at exercise intensities: a comparative study of self-monitoring heart rate and physical activity wearable devices. JMIR Mhealth Uhealth. 2017 Mar 16;5(3):e34. doi: 10.2196/mhealth.7043. PubMed DOI PMC

Rosenberger ME, Buman MP, Haskell WL, McConnell MV, Carstensen LL. Twenty-four hours of sleep, sedentary behavior, and physical activity with nine wearable devices. Med Sci Sports Exerc. 2016 Mar;48(3):457–65. doi: 10.1249/MSS.0000000000000778. PubMed DOI PMC

Ferguson T, Rowlands AV, Olds T, Maher C. The validity of consumer-level, activity monitors in healthy adults worn in free-living conditions: a cross-sectional study. Int J Behav Nutr Phys Act. 2015 Mar 27;12(1):42. doi: 10.1186/s12966-015-0201-9. PubMed DOI PMC

Ko PT, Kientz JA, Choe EK, Kay M, Landis CA, Watson NF. Consumer sleep technologies: a review of the landscape. J Clin Sleep Med. 2015 Dec 15;11(12):1455–61. doi: 10.5664/jcsm.5288. doi: 10.5664/jcsm.5288. PubMed DOI PMC

Brooke SM, An H, Kang S, Noble JM, Berg KE, Lee J. Concurrent validity of wearable activity trackers under free-living conditions. J Strength Cond Res. 2017 Apr;31(4):1097–1106. doi: 10.1519/JSC.0000000000001571. PubMed DOI

Wang R, Blackburn G, Desai M, Phelan D, Gillinov L, Houghtaling P, Gillinov M. Accuracy of wrist-worn heart rate monitors. JAMA Cardiol. 2017 Jan 01;2(1):104–106. doi: 10.1001/jamacardio.2016.3340. PubMed DOI

Gillinov S, Etiwy M, Wang R, Blackburn G, Phelan D, Gillinov AM, Houghtaling P, Javadikasgari H, Desai MY. Variable accuracy of wearable heart rate monitors during aerobic exercise. Med Sci Sports Exerc. 2017 Aug;49(8):1697–1703. doi: 10.1249/MSS.0000000000001284. PubMed DOI

Murakami H, Kawakami R, Nakae S, Nakata Y, Ishikawa-Takata K, Tanaka S, Miyachi M. Accuracy of wearable devices for estimating total energy expenditure: comparison with metabolic chamber and doubly labeled water method. JAMA Intern Med. 2016 May 01;176(5):702–3. doi: 10.1001/jamainternmed.2016.0152. PubMed DOI

Clinical Trials Transformation Initiative. 2018. [2019-05-30]. CTTI Recommendations: Advancing the Use of Mobile Technologies for Data Capture & Improved Clinical Trials https://www.ctti-clinicaltrials.org/files/mobile-devices-recommendations.pdf.

Gagnon M, Ngangue P, Payne-Gagnon J, Desmartis M. m-Health adoption by healthcare professionals: a systematic review. J Am Med Inform Assoc. 2016 Jan;23(1):212–20. doi: 10.1093/jamia/ocv052. PubMed DOI PMC

Simblett S, Greer B, Matcham F, Curtis H, Polhemus A, Ferrão J, Gamble P, Wykes T. Barriers to and facilitators of engagement with remote measurement technology for managing health: systematic review and content analysis of findings. J Med Internet Res. 2018 Jul 12;20(7):e10480. doi: 10.2196/10480. PubMed DOI PMC

Parker SJ, Jessel S, Richardson JE, Reid MC. Older adults are mobile too!Identifying the barriers and facilitators to older adults' use of mHealth for pain management. BMC Geriatr. 2013 May 06;13:43. doi: 10.1186/1471-2318-13-43. PubMed DOI PMC

Altman M, Huang TT, Breland JY. Design thinking in health care. Prev Chronic Dis. 2018 Sep 27;15:E117. doi: 10.5888/pcd15.180128. PubMed DOI PMC

Donnelly S, Reginatto B, Kearns O, Mc Carthy M, Byrom B, Muehlhausen W, Caulfield B. The burden of a remote trial in a nursing home setting: qualitative study. J Med Internet Res. 2018 Jun 19;20(6):e220. doi: 10.2196/jmir.9638. PubMed DOI PMC

O'Connor S, Hanlon P, O'Donnell CA, Garcia S, Glanville J, Mair FS. Understanding factors affecting patient and public engagement and recruitment to digital health interventions: a systematic review of qualitative studies. BMC Med Inform Decis Mak. 2016 Sep 15;16(1):120. doi: 10.1186/s12911-016-0359-3. PubMed DOI PMC

Byrom B, Watson C, Doll H, Coons SJ, Eremenco S, Ballinger R, Mc Carthy M, Crescioni M, O'Donohoe P, Howry C, ePRO Consortium Selection of and evidentiary considerations for wearable devices and their measurements for use in regulatory decision making: recommendations from the ePRO Consortium. Value Health. 2018 Jun;21(6):631–639. doi: 10.1016/j.jval.2017.09.012. PubMed DOI

Clinical Trials Transformation Initiative. 2018. [2019-05-30]. Framework of Specifications to Consider During Mobile Technology Selection https://www.ctti-clinicaltrials.org/sites/www.ctti-clinicaltrials.org/files/technology-selection-tool.pdf.

Ostrovsky A, Deen N, Simon A, Mate K. Institute for Healthcare Improvement. Cambridge, MA: 2014. [2019-05-30]. A framework for selecting digital health technology: IHI innovation report http://www.ihi.org/resources/Pages/Publications/AFrameworkforSelectingDigitalHealthTechnology.aspx.

Ross J, Stevenson F, Lau R, Murray E. Factors that influence the implementation of e-health: a systematic review of systematic reviews (an update) Implement Sci. 2016 Oct 26;11(1):146. doi: 10.1186/s13012-016-0510-7. PubMed DOI PMC

TransCelerate BioPharma, Inc. 2018. [2019-05-30]. Patient technology implementation framework https://transceleratebiopharmainc.com/patient-technology-implementation-framework/

Mummah SA, Robinson TN, King AC, Gardner CD, Sutton S. IDEAS (Integrate, Design, Assess, and Share): a framework and toolkit of strategies for the development of more effective digital interventions to change health behavior. J Med Internet Res. 2016 Dec 16;18(12):e317. doi: 10.2196/jmir.5927. PubMed DOI PMC

van Gemert-Pijnen JE, Nijland N, van Limburg M, Ossebaard HC, Kelders SM, Eysenbach G, Seydel ER. A holistic framework to improve the uptake and impact of eHealth technologies. J Med Internet Res. 2011 Dec 05;13(4):e111. doi: 10.2196/jmir.1672. PubMed DOI PMC

Schnall R, Rojas M, Bakken S, Brown W, Carballo-Dieguez A, Carry M, Gelaude D, Mosley JP, Travers J. A user-centered model for designing consumer mobile health (mHealth) applications (apps) J Biomed Inform. 2016 Apr;60:243–51. doi: 10.1016/j.jbi.2016.02.002. PubMed DOI PMC

Wilhide CC, Peeples MM, Anthony KRC. Evidence-based mHealth chronic disease mobile app intervention design: development of a framework. JMIR Res Protoc. 2016 Feb 16;5(1):e25. doi: 10.2196/resprot.4838. PubMed DOI PMC

Sadegh SS, Khakshour Saadat P, Sepehri MM, Assadi V. A framework for m-health service development and success evaluation. Int J Med Inform. 2018 Apr;112:123–130. doi: 10.1016/j.ijmedinf.2018.01.003. PubMed DOI

Eslami Andargoli A, Scheepers H, Rajendran D, Sohal A. Health information systems evaluation frameworks: A systematic review. Int J Med Inform. 2017 Jan;97:195–209. doi: 10.1016/j.ijmedinf.2016.10.008. PubMed DOI

Norman D. The Design Of Everyday Things. New York, NY: Basic Books; 2013.

Design Kit. IDEO.org; 2019. [2019-02-28]. What is human-centered design? http://www.designkit.org/human-centered-design.

Design Council News and Opinion. London, UK: Design Council; 2015. [2019-02-27]. What is the framework for innovation? Design Council's evolved Double Diamond https://www.designcouncil.org.uk/news-opinion/design-process-what-double-diamond.

Ries E. The Lean Startup: How Today's Entrepreneurs Use Continuous Innovation To Create Radically Successful Businesses. New York: Crown Business; 2011.

Hevner A. A three cycle view of design science research. Scand J Inf Syst. 2007;19(2):7.

Van Velsen L, Wentzel J, Van Gemert-Pijnen GJE. Designing eHealth that matters via a multidisciplinary requirements development approach. JMIR Res Protoc. 2013 Jun 24;2(1):e21. doi: 10.2196/resprot.2547. PubMed DOI PMC

Standford DSchool. 2018. [2019-02-28]. Design thinking bootleg https://dschool.stanford.edu/resources/design-thinking-bootleg.

Woods L, Cummings E, Duff J, Walker K. Partnering in Digital Health Design: Engaging the Multidisciplinary Team in a Needs Analysis. Stud Health Technol Inform. 2018;252:176–181. PubMed

Woods L, Cummings E, Duff J, Walker K. Design thinking for mHealth application co-design to support heart failure self-management. Stud Health Technol Inform. 2017;241:97–102. PubMed

Bate P, Robert G. Experience-based design: from redesigning the system around the patient to co-designing services with the patient. Qual Saf Health Care. 2006 Oct;15(5):307–10. doi: 10.1136/qshc.2005.016527. PubMed DOI PMC

U.S. Food and Drug Administration. 2018. [2019-05-30]. Patient-focused drug development: collecting comprehensive and representative input, guidance for industry, food and drug administration staff, and other stakeholders https://www.fda.gov/downloads/drugs/guidancecomplianceregulatoryinformation/guidances/ucm610442.pdf.

James Lind Alliance. 2018. [2019-01-23]. The PSPs http://www.jla.nihr.ac.uk/priority-setting-partnerships/

Lewis JR. Psychometric evaluation of the PSSUQ using data from five years of usability studies. International Journal of Human-Computer Interaction. 2002 Sep;14(3-4):463–488. doi: 10.1080/10447318.2002.9669130. DOI

Chin WW, Johnson N, Schwarz A. A fast form approach to measuring technology acceptance and other constructs. MIS Quarterly. 2008;32(4):687. doi: 10.2307/25148867. DOI

Matcham F, Barattieri di San Pietro C, Bulgari V, de Girolamo G, Dobson R, Eriksson H, Folarin AA, Haro JM, Kerz M, Lamers F, Li Q, Manyakov NV, Mohr DC, Myin-Germeys I, Narayan V, Bwjh P, Ranjan Y, Rashid Z, Rintala A, Siddi S, Simblett SK, Wykes T, Hotopf M, RADAR-CNS consortium Remote assessment of disease and relapse in major depressive disorder (RADAR-MDD): a multi-centre prospective cohort study protocol. BMC Psychiatry. 2019 Feb 18;19(1):72. doi: 10.1186/s12888-019-2049-z. PubMed DOI PMC

RADAR-CNS Consortium. 2019. [2019-01-23]. RADAR-CNS: Remote Assessment of Disease and Relapse https://www.radar-cns.org/

RADAR-CNS Consortium. 2019. [2019-02-28]. Patient Advisory Board | RADAR-CNS https://www.radar-cns.org/patient-engagement/patient-advisory-board.

Simblett SK, Evans J, Greer B, Curtis H, Matcham F, Radaelli M, Mulero P, Arévalo MJ, Polhemus A, Ferrao J, Gamble P, Comi G, Wykes T, RADAR-CNS consortium Engaging across dimensions of diversity: A cross-national perspective on mHealth tools for managing relapsing remitting and progressive multiple sclerosis. Mult Scler Relat Disord. 2019 Jul;32:123–132. doi: 10.1016/j.msard.2019.04.020. PubMed DOI

Bruno E, Simblett S, Lang A, Biondi A, Odoi C, Schulze-Bonhage A, Wykes T, Richardson MP, RADAR-CNS Consortium Wearable technology in epilepsy: the views of patients, caregivers, and healthcare professionals. Epilepsy Behav. 2018 Aug;85:141–149. doi: 10.1016/j.yebeh.2018.05.044. PubMed DOI

Simblett S, Matcham F, Siddi S, Bulgari V, Barattieri di San Pietro C, Hortas López J, Ferrão J, Polhemus A, Haro JM, de Girolamo G, Gamble P, Eriksson H, Hotopf M, Wykes T, RADAR-CNS Consortium Barriers to and facilitators of engagement with mHealth technology for remote measurement and management of depression: qualitative analysis. JMIR Mhealth Uhealth. 2019 Jan 30;7(1):e11325. doi: 10.2196/11325. PubMed DOI PMC

DasMahapatra P, Chiauzzi E, Bhalerao R, Rhodes J. Free-living physical activity monitoring in adult US patients with multiple sclerosis using a consumer wearable device. Digit Biomark. 2018 Apr 13;2(1):47–63. doi: 10.1159/000488040. PubMed DOI PMC

Block VJ, Lizée A, Crabtree-Hartman E, Bevan CJ, Graves JS, Bove R, Green AJ, Nourbakhsh B, Tremblay M, Gourraud P-A, Ng MY, Pletcher MJ, Olgin JE, Marcus GM, Allen DD, Cree BAC, Gelfand JM. Continuous daily assessment of multiple sclerosis disability using remote step count monitoring. J Neurol. 2017 Feb;264(2):316–326. doi: 10.1007/s00415-016-8334-6. PubMed DOI PMC

Sasaki JE, Sandroff B, Bamman M, Motl RW. Motion sensors in multiple sclerosis: Narrative review and update of applications. Expert Rev Med Devices. 2017 Nov;14(11):891–900. doi: 10.1080/17434440.2017.1386550. PubMed DOI PMC

Data Visualization for Chronic Neurological and Mental Health Condition Self-management: Systematic Review of User Perspectives