Sleep prevents catastrophic forgetting in spiking neural networks by forming a joint synaptic weight representation
Jazyk angličtina Země Spojené státy americké Médium electronic-ecollection
Typ dokumentu časopisecké články, práce podpořená grantem, Research Support, U.S. Gov't, Non-P.H.S., Research Support, N.I.H., Extramural
Grantová podpora
R01 MH125557
NIMH NIH HHS - United States
R01 NS109553
NINDS NIH HHS - United States
RF1 MH117155
NIMH NIH HHS - United States
PubMed
36399437
PubMed Central
PMC9674146
DOI
10.1371/journal.pcbi.1010628
PII: PCOMPBIOL-D-22-00638
Knihovny.cz E-zdroje
- MeSH
- mozek MeSH
- neuronové sítě * MeSH
- spánek MeSH
- učení * fyziologie MeSH
- Publikační typ
- časopisecké články MeSH
- práce podpořená grantem MeSH
- Research Support, N.I.H., Extramural MeSH
- Research Support, U.S. Gov't, Non-P.H.S. MeSH
Artificial neural networks overwrite previously learned tasks when trained sequentially, a phenomenon known as catastrophic forgetting. In contrast, the brain learns continuously, and typically learns best when new training is interleaved with periods of sleep for memory consolidation. Here we used spiking network to study mechanisms behind catastrophic forgetting and the role of sleep in preventing it. The network could be trained to learn a complex foraging task but exhibited catastrophic forgetting when trained sequentially on different tasks. In synaptic weight space, new task training moved the synaptic weight configuration away from the manifold representing old task leading to forgetting. Interleaving new task training with periods of off-line reactivation, mimicking biological sleep, mitigated catastrophic forgetting by constraining the network synaptic weight state to the previously learned manifold, while allowing the weight configuration to converge towards the intersection of the manifolds representing old and new tasks. The study reveals a possible strategy of synaptic weights dynamics the brain applies during sleep to prevent forgetting and optimize learning.
Zobrazit více v PubMed
French RM. Catastrophic forgetting in connectionist networks. Trends Cogn Sci. 1999;3(4):128–35. doi: 10.1016/s1364-6613(99)01294-2 PubMed DOI
Mccloskey M, Cohen NJ. CATASTROPHIC INTERFERENCE IN CONNECTIONIST NETWORKS: THE SEQUENTIAL LEARNING PROBLEM. The Psychology of Learning and Motivation. 1989;24:109–65.
Ratcliff R. Connectionist models of recognition memory: constraints imposed by learning and forgetting functions. Psychol Rev. 1990;97(2):285–308. doi: 10.1037/0033-295x.97.2.285 PubMed DOI
Hasselmo ME. Avoiding Catastrophic Forgetting. Trends Cogn Sci. 2017;21(6):407–8. doi: 10.1016/j.tics.2017.04.001 PubMed DOI
Hassabis D, Kumaran D, Summerfield C, Botvinick M. Neuroscience-Inspired Artificial Intelligence. Neuron. 2017;95(2):245–58. doi: 10.1016/j.neuron.2017.06.011 PubMed DOI
Kemker R, Abitino A, McClure M, Kanan C. Measuring Catastrophic Forgetting in Neural Networks. arXiv:170802072 [Internet]. 2017.
Kirkpatrick J, Pascanu R, Rabinowitz N, Veness J, Desjardins G, Rusu AA, et al.. Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences. 2017;114(13):3521–6. doi: 10.1073/pnas.1611835114 PubMed DOI PMC
Kemker R, Kanan C. Fearnet: Brain-inspired model for incremental learning. arXiv:171110563. 2017.
Hayes TL, Krishnan GP, Bazhenov M, Siegelmann HT, Sejnowski TJ, Kanan C. Replay in Deep Learning: Current Approaches and Missing Biological Elements. Neural computation. 2021;33(11):2908–50. doi: 10.1162/neco_a_01433 PubMed DOI PMC
Flesch T, Balaguer J, Dekker R, Nili H, Summerfield C. Comparing continual task learning in minds and machines. Proc Natl Acad Sci U S A. 2018;115(44):E10313–E22. doi: 10.1073/pnas.1800755115 PubMed DOI PMC
McClelland JL, McNaughton BL, O’Reilly RC. Why there are complementary learning systems in the hippocampus and neocortex: insights from the successes and failures of connectionist models of learning and memory. Psychol Rev. 1995;102(3):419–57. doi: 10.1037/0033-295X.102.3.419 PubMed DOI
Evans BD, Stringer SM. Transformation-invariant visual representations in self-organizing spiking neural networks. Front Comput Neurosci. 2012;6:46. doi: 10.3389/fncom.2012.00046 PubMed DOI PMC
Higgins I, Stringer S, Schnupp J. Unsupervised learning of temporal features for word categorization in a spiking neural network model of the auditory brain. PLoS One. 2017;12(8):e0180174. doi: 10.1371/journal.pone.0180174 PubMed DOI PMC
Sanda P, Skorheim S, Bazhenov M. Multi-layer network utilizing rewarded spike time dependent plasticity to learn a foraging task. PLoS Comput Biol. 2017;13(9):e1005705. doi: 10.1371/journal.pcbi.1005705 PubMed DOI PMC
Skorheim S, Lonjers P, Bazhenov M. A spiking network model of decision making employing rewarded STDP. PLoS One. 2014;9(3):e90821. doi: 10.1371/journal.pone.0090821 PubMed DOI PMC
Rasch B, Born J. About sleep’s role in memory. Physiological reviews. 2013;93(2):681–766. doi: 10.1152/physrev.00032.2012 PubMed DOI PMC
Ji D, Wilson MA. Coordinated memory replay in the visual cortex and hippocampus during sleep. Nat Neurosci. 2007;10(1):100–7. doi: 10.1038/nn1825 PubMed DOI
Euston DR, Tatsuno M, McNaughton BL. Fast-forward playback of recent memory sequences in prefrontal cortex during sleep. Science. 2007;318(5853):1147–50. doi: 10.1126/science.1148979 PubMed DOI
Peyrache A, Khamassi M, Benchenane K, Wiener SI, Battaglia FP. Replay of rule-learning related neural patterns in the prefrontal cortex during sleep. Nat Neurosci. 2009;12(7):919–26. doi: 10.1038/nn.2337 PubMed DOI
Barnes DC, Wilson DA. Slow-wave sleep-imposed replay modulates both strength and precision of memory. J Neurosci. 2014;34(15):5134–42. doi: 10.1523/JNEUROSCI.5274-13.2014 PubMed DOI PMC
Mednick SC, Cai DJ, Shuman T, Anagnostaras S, Wixted JT. An opportunistic theory of cellular and systems consolidation. Trends Neurosci. 2011;34(10):504–14. doi: 10.1016/j.tins.2011.06.003 PubMed DOI PMC
Stickgold R. Parsing the role of sleep in memory processing. Curr Opin Neurobiol. 2013;23(5):847–53. doi: 10.1016/j.conb.2013.04.002 PubMed DOI PMC
Ramanathan DS, Gulati T, Ganguly K. Sleep-Dependent Reactivation of Ensembles in Motor Cortex Promotes Skill Consolidation. PLOS Biology. 2015;13(9):e1002263. doi: 10.1371/journal.pbio.1002263 PubMed DOI PMC
Zwaka H, Bartels R, Gora J, Franck V, Culo A, Gotsch M, et al.. Context odor presentation during sleep enhances memory in honeybees. Curr Biol. 2015;25(21):2869–74. doi: 10.1016/j.cub.2015.09.069 PubMed DOI
Melnattur K, Kirszenblat L, Morgan E, Militchin V, Sakran B, English D, et al.. A conserved role for sleep in supporting Spatial Learning in Drosophila. Sleep. 2021;44(3). doi: 10.1093/sleep/zsaa197 PubMed DOI PMC
Donlea JM, Thimgan MS, Suzuki Y, Gottschalk L, Shaw PJ. Inducing sleep by remote control facilitates memory consolidation in Drosophila. Science. 2011;332(6037):1571–6. doi: 10.1126/science.1202249 PubMed DOI PMC
Cassenaer S, Laurent G. Hebbian STDP in mushroom bodies facilitates the synchronous flow of olfactory information in locusts. Nature. 2007;448(7154):709–13. doi: 10.1038/nature05973 PubMed DOI
Bi GQ, Poo MM. Synaptic modifications in cultured hippocampal neurons: dependence on spike timing, synaptic strength, and postsynaptic cell type. J Neurosci. 1998;18(24):10464–72. doi: 10.1523/JNEUROSCI.18-24-10464.1998 PubMed DOI PMC
Markram H, Lubke J, Frotscher M, Sakmann B. Regulation of synaptic efficacy by coincidence of postsynaptic APs and EPSPs. Science. 1997;275(5297):213–5. doi: 10.1126/science.275.5297.213 PubMed DOI
Farries MA, Fairhall AL. Reinforcement learning with modulated spike timing dependent synaptic plasticity. J Neurophysiol. 2007;98(6):3648–65. doi: 10.1152/jn.00364.2007 PubMed DOI
Florian RV. Reinforcement learning through modulation of spike-timing-dependent synaptic plasticity. Neural Comput. 2007;19(6):1468–502. doi: 10.1162/neco.2007.19.6.1468 PubMed DOI
Izhikevich EM. Solving the distal reward problem through linkage of STDP and dopamine signaling. Cereb Cortex. 2007;17(10):2443–52. doi: 10.1093/cercor/bhl152 PubMed DOI
Legenstein R, Pecevski D, Maass W. A learning theory for reward-modulated spike-timing-dependent plasticity with application to biofeedback. PLoS Comput Biol. 2008;4(10):e1000180. doi: 10.1371/journal.pcbi.1000180 PubMed DOI PMC
LeCun Y, Bengio Y, Hinton G. Deep learning. Nature. 2015;521(7553):436–44. doi: 10.1038/nature14539 PubMed DOI
Cadieu CF, Hong H, Yamins DL, Pinto N, Ardila D, Solomon EA, et al.. Deep neural networks rival the representation of primate IT cortex for core visual object recognition. PLoS Comput Biol. 2014;10(12):e1003963. doi: 10.1371/journal.pcbi.1003963 PubMed DOI PMC
Yamins DL, DiCarlo JJ. Using goal-driven deep learning models to understand sensory cortex. Nat Neurosci. 2016;19(3):356–65. doi: 10.1038/nn.4244 PubMed DOI
Yamins DL, Hong H, Cadieu CF, Solomon EA, Seibert D, DiCarlo JJ. Performance-optimized hierarchical models predict neural responses in higher visual cortex. Proc Natl Acad Sci U S A. 2014;111(23):8619–24. doi: 10.1073/pnas.1403112111 PubMed DOI PMC
Wei Y, Krishnan G, Bazhenov M. Synaptic Mechanisms of Memory Consolidation during Sleep Slow Oscillations. Journal of Neuroscience. 2016;36(15):4231–47. doi: 10.1523/JNEUROSCI.3648-15.2016 PubMed DOI PMC
Wei Y, Krishnan GP, Komarov M, Bazhenov M. Differential roles of sleep spindles and sleep slow oscillations in memory consolidation. PLoS Comput Biol. 2018;14(7):e1006322. doi: 10.1371/journal.pcbi.1006322 PubMed DOI PMC
Wei Y, Krishnan GP, Marshall L, Martinetz T, Bazhenov M. Stimulation Augments Spike Sequence Replay and Memory Consolidation during Slow-Wave Sleep. The Journal of neuroscience: the official journal of the Society for Neuroscience. 2020;40(4):811–24. doi: 10.1523/JNEUROSCI.1427-19.2019 PubMed DOI PMC
Gonzalez OC, Sokolov Y, Krishnan GP, Delanois JE, Bazhenov M. Can sleep protect memories from catastrophic forgetting? Elife. 2020;9. doi: 10.7554/eLife.51005 PubMed DOI PMC
Peever J, Fuller PM. The Biology of REM Sleep. Curr Biol. 2017;27(22):R1237–R48. doi: 10.1016/j.cub.2017.10.026 PubMed DOI
Oudiette D, Antony JW, Creery JD, Paller KA. The role of memory reactivation during wakefulness and sleep in determining which memories endure. J Neurosci. 2013;33(15):6672–8. doi: 10.1523/JNEUROSCI.5497-12.2013 PubMed DOI PMC
Paller KA, Voss JL. Memory reactivation and consolidation during sleep. Learn Mem. 2004;11(6):664–70. doi: 10.1101/lm.75704 PubMed DOI PMC
Walker MP, Stickgold R. Sleep-dependent learning and memory consolidation. Neuron. 2004;44(1):121–33. doi: 10.1016/j.neuron.2004.08.031 PubMed DOI
Stickgold R, James L, Hobson JA. Visual discrimination learning requires sleep after training. Nat Neurosci. 2000;3(12):1237–8. doi: 10.1038/81756 PubMed DOI
Hennevin E, Hars B, Maho C, Bloch V. Processing of learned information in paradoxical sleep: relevance for memory. Behav Brain Res. 1995;69(1–2):125–35. doi: 10.1016/0166-4328(95)00013-j PubMed DOI
Lewis PA, Knoblich G, Poe G. How Memory Replay in Sleep Boosts Creative Problem-Solving. Trends Cogn Sci. 2018;22(6):491–503. doi: 10.1016/j.tics.2018.03.009 PubMed DOI PMC
Oudiette D, Paller KA. Upgrading the sleeping brain with targeted memory reactivation. Trends Cogn Sci. 2013;17(3):142–9. doi: 10.1016/j.tics.2013.01.006 PubMed DOI
McDevitt EA, Duggan KA, Mednick SC. REM sleep rescues learning from interference. Neurobiol Learn Mem. 2015;122:51–62. doi: 10.1016/j.nlm.2014.11.015 PubMed DOI PMC
Javed K, White M. Meta-Learning Representations for Continual Learning. arXiv e-prints [Internet]. 2019. May 01, 2019:[arXiv:1905.12588 p.]. Available from: https://ui.adsabs.harvard.edu/abs/2019arXiv190512588J.
Roumis DK, Frank LM. Hippocampal sharp-wave ripples in waking and sleeping states. Curr Opin Neurobiol. 2015;35:6–12. doi: 10.1016/j.conb.2015.05.001 PubMed DOI PMC
Swanson RA, Levenstein D, McClain K, Tingley D, Buzsáki G. Variable specificity of memory trace reactivation during hippocampal sharp wave ripples. Current Opinion in Behavioral Sciences. 2020;32:126–35. doi: 10.1016/j.cobeha.2020.02.008 PubMed DOI PMC
Krishnan GP, Tadros T, Ramyaa R, Bazhenov M. Biologically inspired sleep algorithm for artificial neural networks. arXiv. 2019:1908.02240v1.
Tadros T, Krishnan G, Ramyaa R, Bazhenov M. Biologically inspired sleep algorithm for reducing catastrophic forgetting in neural networks. AAAI Conference on Artificial Intelligence 2020. p. 13933–4.
Tadros T, Krishnan GP, Ramyaa R, Bazhenov M. Biologically inspired sleep algorithm for increased generalization and adversarial robustness in deep neural networks. International Conference on Learning Representations [Internet]. 2019.
Laurent G. Olfactory network dynamics and the coding of multidimensional signals. Nat Rev Neurosci. 2002;3(11):884–95. doi: 10.1038/nrn964 PubMed DOI
Assisi C, Stopfer M, Laurent G, Bazhenov M. Adaptive regulation of sparseness by feedforward inhibition. Nature neuroscience. 2007;10(9):1176–84. doi: 10.1038/nn1947 PubMed DOI PMC
Perez-Orive J, Bazhenov M, Laurent G. Intrinsic and circuit properties favor coincidence detection for decoding oscillatory input. The Journal of neuroscience: the official journal of the Society for Neuroscience. 2004;24(26):6037–47. PubMed PMC
Schultz W. Dopamine reward prediction-error signalling: a two-component response. Nat Rev Neurosci. 2016;17(3):183–95. doi: 10.1038/nrn.2015.26 PubMed DOI PMC
Schultz W. Dopamine reward prediction error coding. Dialogues Clin Neurosci. 2016;18(1):23–32. doi: 10.31887/DCNS.2016.18.1/wschultz PubMed DOI PMC
Schultz W, Dayan P, Montague PR. A neural substrate of prediction and reward. Science. 1997;275(5306):1593–9. doi: 10.1126/science.275.5306.1593 PubMed DOI
Cassenaer S, Laurent G. Conditional modulation of spike-timing-dependent plasticity for olfactory learning. Nature. 2012;482(7383):47–52. doi: 10.1038/nature10776 PubMed DOI
Tainton-Heap LAL, Kirszenblat LC, Notaras ET, Grabowska MJ, Jeans R, Feng K, et al.. A Paradoxical Kind of Sleep in Drosophila melanogaster. Curr Biol. 2021;31(3):578–90 e6. doi: 10.1016/j.cub.2020.10.081 PubMed DOI PMC
Kaiser W, Steiner-Kaiser J. Neuronal correlates of sleep, wakefulness and arousal in a diurnal insect. Nature. 1983;301(5902):707–9. doi: 10.1038/301707a0 PubMed DOI
Sauer S, Kinkelin M, Herrmann E, Kaiser W. The dynamics of sleep-like behaviour in honey bees. Journal of comparative physiology A, Neuroethology, sensory, neural, and behavioral physiology. 2003;189(8):599–607. doi: 10.1007/s00359-003-0436-9 PubMed DOI
Rulkov NF, Bazhenov M. Oscillations and synchrony in large-scale cortical network models. J Biol Phys. 2008;34(3–4):279–99. doi: 10.1007/s10867-008-9079-y PubMed DOI PMC
Rulkov NF, Timofeev I, Bazhenov M. Oscillations in large-scale cortical networks: map-based model. J Comput Neurosci. 2004;17(2):203–23. doi: 10.1023/B:JCNS.0000037683.55688.7e PubMed DOI
Bazhenov M, Stopfer M. Forward and back: motifs of inhibition in olfactory processing. Neuron. 2010;67(3):357–8. doi: 10.1016/j.neuron.2010.07.023 PubMed DOI PMC
Bruno RM. Synchrony in sensation. Curr Opin Neurobiol. 2011;21(5):701–8. doi: 10.1016/j.conb.2011.06.003 PubMed DOI PMC
Dong H, Shao Z, Nerbonne JM, Burkhalter A. Differential depression of inhibitory synaptic responses in feedforward and feedback circuits between different areas of mouse visual cortex. J Comp Neurol. 2004;475(3):361–73. doi: 10.1002/cne.20164 PubMed DOI
Pouille F, Scanziani M. Enforcement of temporal fidelity in pyramidal cells by somatic feed-forward inhibition. Science. 2001;293(5532):1159–63. doi: 10.1126/science.1060342 PubMed DOI
Shao Z, Burkhalter A. Different balance of excitation and inhibition in forward and feedback circuits of rat visual cortex. Journal of Neuroscience. 1996;16(22):7353–65. doi: 10.1523/JNEUROSCI.16-22-07353.1996 PubMed DOI PMC
Silberberg G. Polysynaptic subcircuits in the neocortex: spatial and temporal diversity. Curr Opin Neurobiol. 2008;18(3):332–7. doi: 10.1016/j.conb.2008.08.009 PubMed DOI
Bazhenov M, Rulkov NF, Fellous JM, Timofeev I. Role of network dynamics in shaping spike timing reliability. Phys Rev E Stat Nonlin Soft Matter Phys. 2005;72(4 Pt 1):041903. doi: 10.1103/PhysRevE.72.041903 PubMed DOI
Rulkov NF. Modeling of spiking-bursting neural behavior using two-dimensional map. Phys Rev E Stat Nonlin Soft Matter Phys. 2002;65(4 Pt 1):041922. doi: 10.1103/PhysRevE.65.041922 PubMed DOI
Komarov M, Krishnan G, Chauvette S, Rulkov N, Timofeev I, Bazhenov M. New class of reduced computationally efficient neuronal models for large-scale simulations of brain dynamics. J Comput Neurosci. 2018;44(1):1–24. doi: 10.1007/s10827-017-0663-7 PubMed DOI