Objective. Virtual reality (VR) has become a key tool for researching spatial memory. Virtual environments offer many advantages for research in terms of logistics, neuroimaging compatibility etc. However, it is well established in animal models that the lack of physical movement in VR impairs some neural representations of space, and this is considered likely to be true in humans as well. Furthermore, it is unclear how big the disruptive effect stationary navigation is-how much does physical movement during encoding and recall affect human spatial memory and representations of space? What effect does the fatigue of actually walking during tasks have on participants-will physical movement decrease performance, or increase perception of difficulty?Approach. We utilize Augmented reality (AR) to enable participants to perform a spatial memory task while physically moving in the real world, compared to a matched VR task performed while stationary. Our task was performed by a group of healthy participants, by a group of stationary epilepsy patients, as they represent the population from which invasive human spatial signals are typically collected, and, in a case study, by a mobile epilepsy patient with an investigational chronic neural implant (Medtronic Summit RC + STM) streaming real-time continuous hippocampal local field potential data.Main results. Participants showed good performance in both conditions, but reported that the walking condition was significantly easier, more immersive, and more fun than the stationary condition. Importantly, memory performance was significantly better in walking vs. stationary in all groups, including epilepsy patients. We also found evidence for an increase in the amplitude of the theta oscillations associated with movement during the walking condition.Significance. Our findings highlight the importance of paradigms that include physical movement and suggest that integrating AR with movement in real environments can lead to improved techniques for spatial memory research.

Czech Institute of Informatics Robotics and Cybernetics Czech Technical University Prague Prague Czech Republic

Department of Biomedical Engineering Ben Gurion University Beer Sheva Israel

Department of Biomedical Engineering Columbia University New York NY United States of America

Department of Neurologic Surgery Mayo Clinic Rochester MN United States of America

Department of Neurological Surgery Columbia University New York NY United States of America

Department of Neurology Mayo Clinic Rochester MN United States of America

Faculty of Biomedical Engineering Czech Technical University Prague Kladno Czech Republic

School of Brain Sciences Ben Gurion University Beer Sheva Israel

Zobrazit více v PubMed

Montana J I, Tuena C, Serino S, Cipresso P, Riva G. Neurorehabilitation of spatial memory using virtual environments: a systematic review. J. Clin. Med. 2019;8:1516. doi: 10.3390/jcm8101516. PubMed DOI PMC

Jonson M, Avramescu S, Chen D, Alam F. The role of virtual reality in screening, diagnosing, and rehabilitating spatial memory deficits. Front. Hum. Neurosci. 2021;15:628818. doi: 10.3389/fnhum.2021.628818. PubMed DOI PMC

Rose F D, Brooks B M, Rizzo A A. Virtual reality in brain damage rehabilitation: review. Cyberpsychol. Behav. 2005;8:241–62. doi: 10.1089/cpb.2005.8.241. PubMed DOI

Starrett M J, Ekstrom A D. Perspective: assessing the flexible acquisition, integration, and deployment of human spatial representations and information. Front. Hum. Neurosci. 2018;12:281. doi: 10.3389/fnhum.2018.00281. PubMed DOI PMC

Aghajan Z M, Acharya L, Moore J J, Cushman J D, Vuong C, Mehta M R. Impaired spatial selectivity and intact phase precession in two-dimensional virtual reality. Nat. Neurosci. 2015;18:121–8. doi: 10.1038/nn.3884. PubMed DOI

Chen G, King J A, Lu Y, Cacucci F, Burgess N. Spatial cell firing during virtual navigation of open arenas by head-restrained mice. elife. 2018;7:e34789. doi: 10.7554/eLife.34789. PubMed DOI PMC

Althoff T, White R W, Horvitz E. Influence of Pokémon Go on physical activity: study and implications. J. Med. Internet Res. 2016;18:e6759. doi: 10.2196/jmir.6759. PubMed DOI PMC

Kangsoo K, Billinghurst M, Bruder G, Duh H, Welch G. Revisiting trends in augmented reality research: a review of the 2nd decade of ISMAR (2008–2017) IEEE Trans.Vis. Comput. Graph. 2018;24:2947–62. doi: 10.1109/TVCG.2018.2868591. PubMed DOI

Milgram P, Kishino F. A taxonomy of mixed reality visual displays. IEICE TRANSACTIONS on Information. 1994;12:1321–9.

Furió D, González-Gancedo S, Juan M-C, Seguí I, Costa M. The effects of the size and weight of a mobile device on an educational game. Comput. Educ. 2013;64:24–41. doi: 10.1016/j.compedu.2012.12.015. DOI

Khademi M, Hondori H M, Dodakian L, Cramer S, Lopes C V. Comparing ‘pick and place’ task in spatial augmented reality versus non-immersive virtual reality for rehabilitation setting. 2013 35th Annual Int. Conf. of the IEEE Engineering in Medicine and Biology Society (EMBC); Osaka, Japan. 2013. pp. 4613–6. PubMed DOI

Mousavi Hondori H, Khademi M, Dodakian L, Cramer S C, Lopes C V. Medicine Meets Virtual Reality 20. IOS Press; 2013. A spatial augmented reality rehab system for post-stroke hand rehabilitation; pp. 279–85. PubMed

Jacobs J, et al. Direct recordings of grid-like neuronal activity in human spatial navigation. Nat. Neurosci. 2013;16:1188–90. doi: 10.1038/nn.3466. PubMed DOI PMC

Tsitsiklis M, et al. Single-neuron representations of spatial targets in humans. Curr. Biol. 2020;30:245–53.e4. doi: 10.1016/j.cub.2019.11.048. PubMed DOI PMC

Miller J, et al. Lateralized hippocampal oscillations underlie distinct aspects of human spatial memory and navigation. Nat. Commun. 2018;9:2423. doi: 10.1038/s41467-018-04847-9. PubMed DOI PMC

Hasselmo M E. What is the function of hippocampal theta rhythm?—Linking behavioral data to phasic properties of field potential and unit recording data. Hippocampus. 2005;15:936–49. doi: 10.1002/hipo.20116. PubMed DOI

Ekstrom A D, Caplan J B, Ho E, Shattuck K, Fried I, Kahana M J. Human hippocampal theta activity during virtual navigation. Hippocampus. 2005;15:881–9. doi: 10.1002/hipo.20109. PubMed DOI

Aghajan Z M, et al. Theta oscillations in the human medial temporal lobe during real-world ambulatory movement. Curr. Biol. 2017;27:3743–51.e3. doi: 10.1016/j.cub.2017.10.062. PubMed DOI PMC

Bohbot V D, Copara M S, Gotman J, Ekstrom A D. Low-frequency theta oscillations in the human hippocampus during real-world and virtual navigation. Nat. Commun. 2017;8:1. doi: 10.1038/ncomms14415. PubMed DOI PMC

Maidenbaum S, Patel A, Stein E, Jacobs J. Spatial memory rehabilitation in virtual reality–extending findings from epilepsy patients to the general population. 2019 Int. Conf. on Virtual Rehabilitation (ICVR); Aviv, Israel. 2019. pp. 1–7. DOI

Maidenbaum S, Miller J, Stein J M, Jacobs J. Grid-like hexadirectional modulation of human entorhinal theta oscillations. Proc. Natl Acad. Sci. 2018;115:10798–803. doi: 10.1073/pnas.1805007115. PubMed DOI PMC

Nowacki P, Woda M. Capabilities of ARCore and ARKit platforms for AR/VR applications. In: Zamojski W, Mazurkiewicz J, Sugier J, Walkowiak T, Kacprzyk J, editors. Engineering in Dependability of Computer Systems and Networks. Springer; 2020. pp. 358–70. Advances in Intelligent Systems and Computing . DOI

Kim P, Kim J, Song M, Lee Y, Jung M, Kim H-G. A benchmark comparison of four off-the-shelf proprietary visual–inertial odometry systems. Sensors. 2022;22:9873. doi: 10.3390/s22249873. PubMed DOI PMC

Kremen V, et al. Integrating brain implants with local and distributed computing devices: a next generation epilepsy management system. IEEE J. Transl. Eng. Health Med. 2018;6:1–12. doi: 10.1109/JTEHM.2018.2869398. PubMed DOI PMC

Gregg N M, et al. Anterior nucleus of the thalamus seizure detection in ambulatory humans. Epilepsia. 2021;62:e158–e64. doi: 10.1111/epi.17047. PubMed DOI PMC

Gilron R, et al. Long-term wireless streaming of neural recordings for circuit discovery and adaptive stimulation in individuals with Parkinson’s disease. Nat. Biotechnol. 2021;39:9. doi: 10.1038/s41587-021-00897-5. PubMed DOI PMC

Pal Attia T, et al. Epilepsy personal assistant device—a mobile platform for brain state, dense behavioral and physiology tracking and controlling adaptive stimulation. Front. Neurol. 2021;12:704170. doi: 10.3389/fneur.2021.704170. PubMed DOI PMC

Hegarty M, Richardson A E, Montello D R, Lovelace K, Subbiah I. Development of a self-report measure of environmental spatial ability. Intelligence. 2002;30:425–47. doi: 10.1016/S0160-2896(02)00116-2. DOI

Hegarty M, Waller D. A dissociation between mental rotation and perspective-taking spatial abilities. Intelligence. 2004;32:175–91. doi: 10.1016/j.intell.2003.12.001. DOI

Condon D M, Wilt J, Cohen C A, Revelle W, Hegarty M, Uttal D H. Sense of direction: general factor saturation and associations with the Big-Five traits. Pers. Individ. Differ. 2015;86:38–43. doi: 10.1016/j.paid.2015.05.023. DOI

Turano K A, Munoz B, Hassan S E, Duncan D D, Gower E W, Roche K B, Keay L, Munro C A, West S K. Poor sense of direction is associated with constricted driving space in older drivers. J. Gerontol. B. 2009;64:348–55. doi: 10.1093/geronb/gbp017. PubMed DOI PMC

Juan M, Furio D, Alem L, Ashworth P, Cano J. DSpace at University of West Bohemia: aRGreenet and BasicGreenet: two mobile games for learning how to recycle. The 19th Int. Conf. in Central Europe on Computer Graphics, Visualization and Computer; 2011.2011.

Furio D, Gnzalez-Gancedo S, Carmen J, Segui I, Rando N. Evaluation of learning outcomes using an educational iPhone game vs. traditional game—ScienceDirect. Comput. Educ. 2012;64:1–23.

Albrecht U-V, Folta-Schoofs K, Behrends M, Jan U V. Effects of mobile augmented reality learning compared to textbook learning on medical students: randomized controlled pilot study. J. Med. Internet Res. 2013;15:e182. doi: 10.2196/jmir.2497. PubMed DOI PMC

Liu P E, Tsai M. Using augmented‐reality‐based mobile learning material in EFL English composition: an exploratory case study. Br. J. Educ. Technol. 2013;44:E1–E4. doi: 10.1111/j.1467-8535.2012.01302.x. DOI

Montello D R, Waller D, Hegarty M, Richardson A E. Human Spatial Memory. Psychology Press; 2004. pp. 251–85.

Waller D, Hunt E, Knapp D. The transfer of spatial knowledge in virtual environment training. Presence. 1998;7:129–43. doi: 10.1162/105474698565631. DOI

Aronov D, Tank D W. Engagement of neural circuits underlying 2D spatial navigation in a rodent virtual reality system. Neuron. 2014;84:442–56. doi: 10.1016/j.neuron.2014.08.042. PubMed DOI PMC

Yassa M A. Brain rhythms: higher-frequency theta oscillations make sense in moving humans. Curr. Biol. 2018;28:R70–2. doi: 10.1016/j.cub.2017.11.045. PubMed DOI PMC

Ruddle R A, Volkova E, Mohler B, Bülthoff H H. The effect of landmark and body-based sensory information on route knowledge. Mem. Cogn. 2011;39:686–99. doi: 10.3758/s13421-010-0054-z. PubMed DOI

Ruddle R A, Volkova E, Bülthoff H H. Learning to walk in virtual reality. ACM Trans. Appl. Percept. 2013;10:1–17. doi: 10.1145/2465780.2465785. DOI

Paris R, Joshi M, He Q, Narasimham G, McNamara T P, Bodenheimer B. Acquisition of survey knowledge using walking in place and resetting methods in immersive virtual environments. Proc. ACM Symp. on Applied Perception; ACM; 2017. pp. 1–8. DOI

Souman J L, Giordano P R, Frissen I, Luca A D, Ernst M O. Making virtual walking real: perceptual evaluation of a new treadmill control algorithm. ACM Trans. Appl. Percept. 2010;7:1–14. doi: 10.1145/1670671.1670675. DOI

Hejtmanek L, Starrett M, Ferrer E, Ekstrom A D. How much of what we learn in virtual reality transfers to real-world navigation? Multisensory Res. 2020;33:479–503. doi: 10.1163/22134808-20201445. PubMed DOI

Sachs G. Virtual & augmented reality: the next big computing platform. Equity Res. 2016.

Chicchi Giglioli I A, Pallavicini F, Pedroli E, Serino S, Riva G. Augmented reality: a brand new challenge for the assessment and treatment of psychological disorders. Comput. Math. Methods Med. 2015;2015:1–12. doi: 10.1155/2015/862942. PubMed DOI PMC

Koenig S T, Krch D, Lange B S, Rizzo A. Virtual reality and rehabilitation. 2019. (available at: https://psycnet.apa.org/record/2019-25330-032) (Accessed 9 February 2024)

Nincarean D, Alia M B, Halim N D A, Rahman M H A. Mobile augmented reality: the potential for education. Proc. Soc. Behav. Sci. 2013;103:657–64. doi: 10.1016/j.sbspro.2013.10.385. DOI

Vinci C, Brandon K O, Kleinjan M, Brandon T H. The clinical potential of augmented reality. Clin. Psychol. Sci. Pract. 2020;27:110. doi: 10.1111/cpsp.12357. PubMed DOI PMC

Krucoff M O, Wozny T A, Lee A T, Rao V R, Chang E F. Operative technique and lessons learned from surgical implantation of the neuropace responsive neurostimulation® system in 57 consecutive patients. Oper. Neurosurg. 2021;20:E98–109. doi: 10.1093/ons/opaa300. PubMed DOI

Gorman C, Gustafsson L. The use of augmented reality for rehabilitation after stroke: a narrative review. Disabil. Rehabil. Assist. Technol. 2022;17:409–17. doi: 10.1080/17483107.2020.1791264. PubMed DOI

Denche-Zamorano A, Rodriguez-Redondo Y, Barrios-Fernandez S, Mendoza-Muñoz M, Castillo-Paredes A, Rojo-Ramos J, Garcia-Gordillo M A, Adsuar J C. Rehabilitation is the main topic in virtual and augmented reality and physical activity research: a bibliometric analysis. Sensors. 2023;23:6. doi: 10.3390/s23062987. PubMed DOI PMC