BACKGROUND: Mobile Ecological Momentary Assessment (EMA) is increasingly used to gather intensive, longitudinal data on behavioral nutrition, physical activity and sedentary behavior and their underlying determinants. However, a relevant concern is the risk of non-random non-compliance with mobile EMA protocols, especially in older adults. This study aimed to examine older adults' compliance with mobile EMA in health behavior studies according to participant characteristics, and prompt timing, and to provide recommendations for future EMA research. METHODS: Data of four intensive longitudinal observational studies employing mobile EMA to understand health behavior, involving 271 community-dwelling older adults (M = 71.8 years, SD = 6.8; 52% female) in Flanders, were pooled. EMA questionnaires were prompted by a smartphone application during specific time slots or events. Data on compliance (i.e. information whether a participant answered at least one item following the prompt), time slot (morning, afternoon or evening) and day (week or weekend day) of each prompt were extracted from the EMA applications. Participant characteristics, including demographics, body mass index, and smartphone ownership, were collected via self-report. Descriptive statistics of compliance were computed, and logistic mixed models were run to examine inter- and intrapersonal variability in compliance. RESULTS: EMA compliance averaged 77.5%, varying from 70.0 to 86.1% across studies. Compliance differed among subgroups and throughout the day. Age was associated with lower compliance (OR = 0.96, 95%CI = 0.93-0.99), while marital/cohabiting status and smartphone ownership were associated with higher compliance (OR = 1.83, 95%CI = 1.21-2.77, and OR = 4.43, 95%CI = 2.22-8.83, respectively). Compliance was lower in the evening than in the morning (OR = 0.82, 95%CI = 0.69-0.97), indicating non-random patterns that could impact study validity. CONCLUSIONS: The findings of this study shed light on the complexities surrounding compliance with mobile EMA protocols among older adults in health behavior studies. Our analysis revealed that non-compliance within our pooled dataset was not completely random. This non-randomness could introduce bias into study findings, potentially compromising the validity of research findings. To address these challenges, we recommend adopting tailored approaches that take into account individual characteristics and temporal dynamics. Additionally, the utilization of Directed Acyclic Graphs, and advanced statistical techniques can help mitigate the impact of non-compliance on study validity.

Department of Movement and Sport Sciences Faculty of Medicine and Health Sciences Ghent University Watersportlaan 2 Ghent Ghent B 9000 Belgium

Department of Public Health and Primary Care Faculty of Medicine and Health Sciences Ghent University Ghent Belgium

Faculty of Physical Education and Sport Charles University Prague Czech Republic

Research Foundation Flanders Brussels Belgium

Zobrazit více v PubMed

Perski O, et al. Understanding health behaviours in context: a systematic review and meta-analysis of ecological momentary assessment studies of five key health behaviours. Health Psychol Rev. 2022;16(4):576–601. 10.1080/17437199.2022.2112258 PubMed DOI PMC

Dao KP, et al. Smartphone-delivered ecological momentary interventions based on ecological momentary assessments to promote health behaviors: systematic review and adapted checklist for reporting ecological momentary assessment and intervention studies. JMIR mHealth uHealth. 2021;9(11):e22890. 10.2196/22890 PubMed DOI PMC

Degroote L, et al. Content validity and methodological considerations in ecological momentary assessment studies on physical activity and sedentary behaviour: a systematic review. Int J Behav Nutr Phys Activity. 2020;17(1):1–13.10.1186/s12966-020-00932-9 PubMed DOI PMC

Shiffman S, Stone AA, Hufford MR. Ecological momentary assessment. Annu Rev Clin Psychol. 2008;4:1–32. 10.1146/annurev.clinpsy.3.022806.091415 PubMed DOI

Cain AE, Depp CA, Jeste DV. Ecological momentary assessment in aging research: a critical review. J Psychiatr Res. 2009;43(11):987–96. 10.1016/j.jpsychires.2009.01.014 PubMed DOI PMC

Smyth JM, Stone AA. Ecological momentary assessment research in behavioral medicine. J Happiness Stud. 2003;4:35–52.10.1023/A:1023657221954 DOI

Hufford MR. Special methodological challenges and opportunities in ecological momentary assessment. The science of real-time data capture: Self-reports in health research, 2007: pp. 54–75.

Jones A, et al. Compliance with ecological momentary assessment protocols in substance users: a meta-analysis. Addiction. 2019;114(4):609–19. 10.1111/add.14503 PubMed DOI PMC

Williams MT, et al. Compliance with mobile ecological momentary assessment of self-reported health-related behaviors and psychological constructs in adults: systematic review and meta-analysis. J Med Internet Res. 2021;23(3):e17023. 10.2196/17023 PubMed DOI PMC

Wrzus C, Neubauer AB. Ecological momentary assessment: a meta-analysis on designs, samples, and compliance across research fields. Assessment. 2023;30(3):825–46. 10.1177/10731911211067538 PubMed DOI PMC

Yao L, et al. Compliance with ecological momentary assessment programmes in the elderly: a systematic review and meta-analysis. BMJ open. 2023;13(7):e069523. 10.1136/bmjopen-2022-069523 PubMed DOI PMC

Gualtieri L, et al. Digital literacy: a barrier to adoption of connected health technologies in older adults. Iproceedings. 2018;4(2):e11803.10.2196/11803 DOI

Murray AL, et al. Prompt-level predictors of compliance in an ecological momentary assessment study of young adults’ mental health. J Affect Disord. 2023;322:125–31. 10.1016/j.jad.2022.11.014 PubMed DOI

Heitjan DF, Basu S. Distinguishing missing at random and missing completely at random. Am Stat. 1996;50(3):207–13.

Azevedo MR, et al. Gender differences in leisure-time physical activity. Int J Public Health. 2007;52:8–15. 10.1007/s00038-006-5062-1 PubMed DOI PMC

Reiter T, Schoedel R. Never miss a beep: using mobile sensing to investigate (non-) compliance in experience sampling studies. Behav Res Methods, 2023: p. 1–23. PubMed PMC

Maes I, et al. The variability of emotions, physical complaints, intention, and self-efficacy: an ecological momentary assessment study in older adults. PeerJ. 2022;10:e13234. 10.7717/peerj.13234 PubMed DOI PMC

Cnudde K, De Vylder F, Cardon G, Maes I, Van Dyck D. Within- and between-person associations of time-varying determinants with snacking in older adults: an ecological momentary assessment study. Br J Nutrition. 1–31. Published online 2024:1–12. 10.1017/S0007114524001004 PubMed

O’Brien ST et al. SEMA3: a free smartphone platform for daily life surveys. 2023. PubMed PMC

Maes I, et al. Within-person associations of Accelerometer-assessed physical activity with time-varying determinants in older adults: time-based ecological momentary Assessment Study. JMIR Aging. 2023;6:e44425. 10.2196/44425 PubMed DOI PMC

Bosker R, Snijders TA. Multilevel analysis: an introduction to basic and advanced multilevel modeling. Multilevel Anal, 2011: p. 1–368.

Jeong J-H, Bae S-M. The relationship between types of Smartphone Use, Digital Literacy, and Smartphone Addiction in the Elderly. Psychiatry Invest. 2022;19(10):832.10.30773/pi.2021.0400 PubMed DOI PMC

Neves BB, Amaro F, Fonseca JR. Coming of (old) age in the digital age: ICT usage and non-usage among older adults. Sociol Res Online. 2013;18(2):22–35.10.5153/sro.2998 DOI

Van Acker J, Maenhout L, Compernolle S. Older adults’ user Engagement with Mobile Health: a systematic review of qualitative and mixed-methods studies. Innov Aging. 2023;7(2):igad007. 10.1093/geroni/igad007 PubMed DOI PMC

Dzubur E, et al. Response patterns and intra-dyadic factors related to compliance with ecological momentary assessment among mothers and children. Translational Behav Med. 2018;8(2):233–42.10.1093/tbm/ibx002 PubMed DOI PMC

Smyth JM, et al. Influence of ecological momentary assessment study design features on reported willingness to participate and perceptions of potential research studies: an experimental study. BMJ open. 2021;11(7):e049154. 10.1136/bmjopen-2021-049154 PubMed DOI PMC

Delgado-Rodriguez M, Llorca J. Bias J Epidemiol Community Health. 2004;58(8):635. PubMed PMC

Murray A et al. Respondent characteristics associated with adherence in a general population ecological momentary assessment study. International Journal of Methods in Psychiatric Research, 2023: p. e1972. PubMed PMC

Van Cauwenberg J, De Paepe A, Poppe L. Lost without a cause: time to embrace causal thinking using Directed Acyclic Graphs (DAGs). Int J Behav Nutr Phys Activity. 2023;20(1):145.10.1186/s12966-023-01545-8 PubMed DOI PMC

Lu H, et al. Toward a clearer definition of selection bias when estimating causal effects. Epidemiology. 2022;33(5):699–706. 10.1097/EDE.0000000000001516 PubMed DOI PMC

Tennant PW, et al. Use of directed acyclic graphs (DAGs) to identify confounders in applied health research: review and recommendations. Int J Epidemiol. 2021;50(2):620–32. 10.1093/ije/dyaa213 PubMed DOI PMC

Digitale JC, et al. Key concepts in clinical epidemiology: collider-conditioning bias. J Clin Epidemiol. 2023;161:152–6. 10.1016/j.jclinepi.2023.07.004 PubMed DOI

Kovalchik SA, et al. Scaled inverse probability weighting: a method to assess potential bias due to event nonreporting in ecological momentary assessment studies. J Educational Behav Stat. 2018;43(3):354–81.10.3102/1076998617738241 DOI

Ji L, et al. Multiple imputation of missing data in multilevel ecological momentary assessments: an example using smoking cessation study data. Front Digit Health. 2023;5:1099517. 10.3389/fdgth.2023.1099517 PubMed DOI PMC

Ziesemer K, et al. Occurrence of and reasons for missing events in mobile dietary assessments: results from three event-based ecological momentary assessment studies. JMIR mHealth uHealth. 2020;8(10):e15430. 10.2196/15430 PubMed DOI PMC

Sun J, Rhemtulla M, Vazire S. Eavesdropping on missing data: what are university students doing when they miss experience sampling reports? Pers Soc Psychol Bull. 2021;47(11):1535–49. 10.1177/0146167220964639 PubMed DOI

Sensor-triggered ecological momentary assessment in physical activity and sedentary behaviour research among Belgian community-dwelling elderly: lessons learnt from intensive longitudinal studies

Contexts, affective and physical states and their variations during physical activity in older adults: an intensive longitudinal study with sensor-triggered event-based ecological momentary assessments

Ecological momentary assessment of physical and eating behaviours: The WEALTH feasibility and optimisation study with recommendations for large-scale data collection