Artificial neural networks overwrite previously learned tasks when trained sequentially, a phenomenon known as catastrophic forgetting. In contrast, the brain learns continuously, and typically learns best when new training is interleaved with periods of sleep for memory consolidation. Here we used spiking network to study mechanisms behind catastrophic forgetting and the role of sleep in preventing it. The network could be trained to learn a complex foraging task but exhibited catastrophic forgetting when trained sequentially on different tasks. In synaptic weight space, new task training moved the synaptic weight configuration away from the manifold representing old task leading to forgetting. Interleaving new task training with periods of off-line reactivation, mimicking biological sleep, mitigated catastrophic forgetting by constraining the network synaptic weight state to the previously learned manifold, while allowing the weight configuration to converge towards the intersection of the manifolds representing old and new tasks. The study reveals a possible strategy of synaptic weights dynamics the brain applies during sleep to prevent forgetting and optimize learning.

Department of Computer Science and Engineering University of California San Diego La Jolla California United States of America

Department of Medicine University of California San Diego La Jolla California United States of America

Institute of Computer Science of the Czech Academy of Sciences Prague Czech Republic

Neurosciences Graduate Program University of California San Diego La Jolla California United States of America

Zobrazit více v PubMed

French RM. Catastrophic forgetting in connectionist networks. Trends Cogn Sci. 1999;3(4):128–35. doi: 10.1016/s1364-6613(99)01294-2 PubMed DOI

Mccloskey M, Cohen NJ. CATASTROPHIC INTERFERENCE IN CONNECTIONIST NETWORKS: THE SEQUENTIAL LEARNING PROBLEM. The Psychology of Learning and Motivation. 1989;24:109–65.

Ratcliff R. Connectionist models of recognition memory: constraints imposed by learning and forgetting functions. Psychol Rev. 1990;97(2):285–308. doi: 10.1037/0033-295x.97.2.285 PubMed DOI

Hasselmo ME. Avoiding Catastrophic Forgetting. Trends Cogn Sci. 2017;21(6):407–8. doi: 10.1016/j.tics.2017.04.001 PubMed DOI

Hassabis D, Kumaran D, Summerfield C, Botvinick M. Neuroscience-Inspired Artificial Intelligence. Neuron. 2017;95(2):245–58. doi: 10.1016/j.neuron.2017.06.011 PubMed DOI

Kemker R, Abitino A, McClure M, Kanan C. Measuring Catastrophic Forgetting in Neural Networks. arXiv:170802072 [Internet]. 2017.

Kirkpatrick J, Pascanu R, Rabinowitz N, Veness J, Desjardins G, Rusu AA, et al.. Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences. 2017;114(13):3521–6. doi: 10.1073/pnas.1611835114 PubMed DOI PMC

Kemker R, Kanan C. Fearnet: Brain-inspired model for incremental learning. arXiv:171110563. 2017.

Hayes TL, Krishnan GP, Bazhenov M, Siegelmann HT, Sejnowski TJ, Kanan C. Replay in Deep Learning: Current Approaches and Missing Biological Elements. Neural computation. 2021;33(11):2908–50. doi: 10.1162/neco_a_01433 PubMed DOI PMC

Flesch T, Balaguer J, Dekker R, Nili H, Summerfield C. Comparing continual task learning in minds and machines. Proc Natl Acad Sci U S A. 2018;115(44):E10313–E22. doi: 10.1073/pnas.1800755115 PubMed DOI PMC

McClelland JL, McNaughton BL, O’Reilly RC. Why there are complementary learning systems in the hippocampus and neocortex: insights from the successes and failures of connectionist models of learning and memory. Psychol Rev. 1995;102(3):419–57. doi: 10.1037/0033-295X.102.3.419 PubMed DOI

Evans BD, Stringer SM. Transformation-invariant visual representations in self-organizing spiking neural networks. Front Comput Neurosci. 2012;6:46. doi: 10.3389/fncom.2012.00046 PubMed DOI PMC

Higgins I, Stringer S, Schnupp J. Unsupervised learning of temporal features for word categorization in a spiking neural network model of the auditory brain. PLoS One. 2017;12(8):e0180174. doi: 10.1371/journal.pone.0180174 PubMed DOI PMC

Sanda P, Skorheim S, Bazhenov M. Multi-layer network utilizing rewarded spike time dependent plasticity to learn a foraging task. PLoS Comput Biol. 2017;13(9):e1005705. doi: 10.1371/journal.pcbi.1005705 PubMed DOI PMC

Skorheim S, Lonjers P, Bazhenov M. A spiking network model of decision making employing rewarded STDP. PLoS One. 2014;9(3):e90821. doi: 10.1371/journal.pone.0090821 PubMed DOI PMC

Rasch B, Born J. About sleep’s role in memory. Physiological reviews. 2013;93(2):681–766. doi: 10.1152/physrev.00032.2012 PubMed DOI PMC

Ji D, Wilson MA. Coordinated memory replay in the visual cortex and hippocampus during sleep. Nat Neurosci. 2007;10(1):100–7. doi: 10.1038/nn1825 PubMed DOI

Euston DR, Tatsuno M, McNaughton BL. Fast-forward playback of recent memory sequences in prefrontal cortex during sleep. Science. 2007;318(5853):1147–50. doi: 10.1126/science.1148979 PubMed DOI

Peyrache A, Khamassi M, Benchenane K, Wiener SI, Battaglia FP. Replay of rule-learning related neural patterns in the prefrontal cortex during sleep. Nat Neurosci. 2009;12(7):919–26. doi: 10.1038/nn.2337 PubMed DOI

Barnes DC, Wilson DA. Slow-wave sleep-imposed replay modulates both strength and precision of memory. J Neurosci. 2014;34(15):5134–42. doi: 10.1523/JNEUROSCI.5274-13.2014 PubMed DOI PMC

Mednick SC, Cai DJ, Shuman T, Anagnostaras S, Wixted JT. An opportunistic theory of cellular and systems consolidation. Trends Neurosci. 2011;34(10):504–14. doi: 10.1016/j.tins.2011.06.003 PubMed DOI PMC

Stickgold R. Parsing the role of sleep in memory processing. Curr Opin Neurobiol. 2013;23(5):847–53. doi: 10.1016/j.conb.2013.04.002 PubMed DOI PMC

Ramanathan DS, Gulati T, Ganguly K. Sleep-Dependent Reactivation of Ensembles in Motor Cortex Promotes Skill Consolidation. PLOS Biology. 2015;13(9):e1002263. doi: 10.1371/journal.pbio.1002263 PubMed DOI PMC

Zwaka H, Bartels R, Gora J, Franck V, Culo A, Gotsch M, et al.. Context odor presentation during sleep enhances memory in honeybees. Curr Biol. 2015;25(21):2869–74. doi: 10.1016/j.cub.2015.09.069 PubMed DOI

Melnattur K, Kirszenblat L, Morgan E, Militchin V, Sakran B, English D, et al.. A conserved role for sleep in supporting Spatial Learning in Drosophila. Sleep. 2021;44(3). doi: 10.1093/sleep/zsaa197 PubMed DOI PMC

Donlea JM, Thimgan MS, Suzuki Y, Gottschalk L, Shaw PJ. Inducing sleep by remote control facilitates memory consolidation in Drosophila. Science. 2011;332(6037):1571–6. doi: 10.1126/science.1202249 PubMed DOI PMC

Cassenaer S, Laurent G. Hebbian STDP in mushroom bodies facilitates the synchronous flow of olfactory information in locusts. Nature. 2007;448(7154):709–13. doi: 10.1038/nature05973 PubMed DOI

Bi GQ, Poo MM. Synaptic modifications in cultured hippocampal neurons: dependence on spike timing, synaptic strength, and postsynaptic cell type. J Neurosci. 1998;18(24):10464–72. doi: 10.1523/JNEUROSCI.18-24-10464.1998 PubMed DOI PMC

Markram H, Lubke J, Frotscher M, Sakmann B. Regulation of synaptic efficacy by coincidence of postsynaptic APs and EPSPs. Science. 1997;275(5297):213–5. doi: 10.1126/science.275.5297.213 PubMed DOI

Farries MA, Fairhall AL. Reinforcement learning with modulated spike timing dependent synaptic plasticity. J Neurophysiol. 2007;98(6):3648–65. doi: 10.1152/jn.00364.2007 PubMed DOI

Florian RV. Reinforcement learning through modulation of spike-timing-dependent synaptic plasticity. Neural Comput. 2007;19(6):1468–502. doi: 10.1162/neco.2007.19.6.1468 PubMed DOI

Izhikevich EM. Solving the distal reward problem through linkage of STDP and dopamine signaling. Cereb Cortex. 2007;17(10):2443–52. doi: 10.1093/cercor/bhl152 PubMed DOI

Legenstein R, Pecevski D, Maass W. A learning theory for reward-modulated spike-timing-dependent plasticity with application to biofeedback. PLoS Comput Biol. 2008;4(10):e1000180. doi: 10.1371/journal.pcbi.1000180 PubMed DOI PMC

LeCun Y, Bengio Y, Hinton G. Deep learning. Nature. 2015;521(7553):436–44. doi: 10.1038/nature14539 PubMed DOI

Cadieu CF, Hong H, Yamins DL, Pinto N, Ardila D, Solomon EA, et al.. Deep neural networks rival the representation of primate IT cortex for core visual object recognition. PLoS Comput Biol. 2014;10(12):e1003963. doi: 10.1371/journal.pcbi.1003963 PubMed DOI PMC

Yamins DL, DiCarlo JJ. Using goal-driven deep learning models to understand sensory cortex. Nat Neurosci. 2016;19(3):356–65. doi: 10.1038/nn.4244 PubMed DOI

Yamins DL, Hong H, Cadieu CF, Solomon EA, Seibert D, DiCarlo JJ. Performance-optimized hierarchical models predict neural responses in higher visual cortex. Proc Natl Acad Sci U S A. 2014;111(23):8619–24. doi: 10.1073/pnas.1403112111 PubMed DOI PMC

Wei Y, Krishnan G, Bazhenov M. Synaptic Mechanisms of Memory Consolidation during Sleep Slow Oscillations. Journal of Neuroscience. 2016;36(15):4231–47. doi: 10.1523/JNEUROSCI.3648-15.2016 PubMed DOI PMC

Wei Y, Krishnan GP, Komarov M, Bazhenov M. Differential roles of sleep spindles and sleep slow oscillations in memory consolidation. PLoS Comput Biol. 2018;14(7):e1006322. doi: 10.1371/journal.pcbi.1006322 PubMed DOI PMC

Wei Y, Krishnan GP, Marshall L, Martinetz T, Bazhenov M. Stimulation Augments Spike Sequence Replay and Memory Consolidation during Slow-Wave Sleep. The Journal of neuroscience: the official journal of the Society for Neuroscience. 2020;40(4):811–24. doi: 10.1523/JNEUROSCI.1427-19.2019 PubMed DOI PMC

Gonzalez OC, Sokolov Y, Krishnan GP, Delanois JE, Bazhenov M. Can sleep protect memories from catastrophic forgetting? Elife. 2020;9. doi: 10.7554/eLife.51005 PubMed DOI PMC

Peever J, Fuller PM. The Biology of REM Sleep. Curr Biol. 2017;27(22):R1237–R48. doi: 10.1016/j.cub.2017.10.026 PubMed DOI

Oudiette D, Antony JW, Creery JD, Paller KA. The role of memory reactivation during wakefulness and sleep in determining which memories endure. J Neurosci. 2013;33(15):6672–8. doi: 10.1523/JNEUROSCI.5497-12.2013 PubMed DOI PMC

Paller KA, Voss JL. Memory reactivation and consolidation during sleep. Learn Mem. 2004;11(6):664–70. doi: 10.1101/lm.75704 PubMed DOI PMC

Walker MP, Stickgold R. Sleep-dependent learning and memory consolidation. Neuron. 2004;44(1):121–33. doi: 10.1016/j.neuron.2004.08.031 PubMed DOI

Stickgold R, James L, Hobson JA. Visual discrimination learning requires sleep after training. Nat Neurosci. 2000;3(12):1237–8. doi: 10.1038/81756 PubMed DOI

Hennevin E, Hars B, Maho C, Bloch V. Processing of learned information in paradoxical sleep: relevance for memory. Behav Brain Res. 1995;69(1–2):125–35. doi: 10.1016/0166-4328(95)00013-j PubMed DOI

Lewis PA, Knoblich G, Poe G. How Memory Replay in Sleep Boosts Creative Problem-Solving. Trends Cogn Sci. 2018;22(6):491–503. doi: 10.1016/j.tics.2018.03.009 PubMed DOI PMC

Oudiette D, Paller KA. Upgrading the sleeping brain with targeted memory reactivation. Trends Cogn Sci. 2013;17(3):142–9. doi: 10.1016/j.tics.2013.01.006 PubMed DOI

McDevitt EA, Duggan KA, Mednick SC. REM sleep rescues learning from interference. Neurobiol Learn Mem. 2015;122:51–62. doi: 10.1016/j.nlm.2014.11.015 PubMed DOI PMC

Javed K, White M. Meta-Learning Representations for Continual Learning. arXiv e-prints [Internet]. 2019. May 01, 2019:[arXiv:1905.12588 p.]. Available from: https://ui.adsabs.harvard.edu/abs/2019arXiv190512588J.

Roumis DK, Frank LM. Hippocampal sharp-wave ripples in waking and sleeping states. Curr Opin Neurobiol. 2015;35:6–12. doi: 10.1016/j.conb.2015.05.001 PubMed DOI PMC

Swanson RA, Levenstein D, McClain K, Tingley D, Buzsáki G. Variable specificity of memory trace reactivation during hippocampal sharp wave ripples. Current Opinion in Behavioral Sciences. 2020;32:126–35. doi: 10.1016/j.cobeha.2020.02.008 PubMed DOI PMC

Krishnan GP, Tadros T, Ramyaa R, Bazhenov M. Biologically inspired sleep algorithm for artificial neural networks. arXiv. 2019:1908.02240v1.

Tadros T, Krishnan G, Ramyaa R, Bazhenov M. Biologically inspired sleep algorithm for reducing catastrophic forgetting in neural networks. AAAI Conference on Artificial Intelligence 2020. p. 13933–4.

Tadros T, Krishnan GP, Ramyaa R, Bazhenov M. Biologically inspired sleep algorithm for increased generalization and adversarial robustness in deep neural networks. International Conference on Learning Representations [Internet]. 2019.

Laurent G. Olfactory network dynamics and the coding of multidimensional signals. Nat Rev Neurosci. 2002;3(11):884–95. doi: 10.1038/nrn964 PubMed DOI

Assisi C, Stopfer M, Laurent G, Bazhenov M. Adaptive regulation of sparseness by feedforward inhibition. Nature neuroscience. 2007;10(9):1176–84. doi: 10.1038/nn1947 PubMed DOI PMC

Perez-Orive J, Bazhenov M, Laurent G. Intrinsic and circuit properties favor coincidence detection for decoding oscillatory input. The Journal of neuroscience: the official journal of the Society for Neuroscience. 2004;24(26):6037–47. PubMed PMC

Schultz W. Dopamine reward prediction-error signalling: a two-component response. Nat Rev Neurosci. 2016;17(3):183–95. doi: 10.1038/nrn.2015.26 PubMed DOI PMC

Schultz W. Dopamine reward prediction error coding. Dialogues Clin Neurosci. 2016;18(1):23–32. doi: 10.31887/DCNS.2016.18.1/wschultz PubMed DOI PMC

Schultz W, Dayan P, Montague PR. A neural substrate of prediction and reward. Science. 1997;275(5306):1593–9. doi: 10.1126/science.275.5306.1593 PubMed DOI

Cassenaer S, Laurent G. Conditional modulation of spike-timing-dependent plasticity for olfactory learning. Nature. 2012;482(7383):47–52. doi: 10.1038/nature10776 PubMed DOI

Tainton-Heap LAL, Kirszenblat LC, Notaras ET, Grabowska MJ, Jeans R, Feng K, et al.. A Paradoxical Kind of Sleep in Drosophila melanogaster. Curr Biol. 2021;31(3):578–90 e6. doi: 10.1016/j.cub.2020.10.081 PubMed DOI PMC

Kaiser W, Steiner-Kaiser J. Neuronal correlates of sleep, wakefulness and arousal in a diurnal insect. Nature. 1983;301(5902):707–9. doi: 10.1038/301707a0 PubMed DOI

Sauer S, Kinkelin M, Herrmann E, Kaiser W. The dynamics of sleep-like behaviour in honey bees. Journal of comparative physiology A, Neuroethology, sensory, neural, and behavioral physiology. 2003;189(8):599–607. doi: 10.1007/s00359-003-0436-9 PubMed DOI

Rulkov NF, Bazhenov M. Oscillations and synchrony in large-scale cortical network models. J Biol Phys. 2008;34(3–4):279–99. doi: 10.1007/s10867-008-9079-y PubMed DOI PMC

Rulkov NF, Timofeev I, Bazhenov M. Oscillations in large-scale cortical networks: map-based model. J Comput Neurosci. 2004;17(2):203–23. doi: 10.1023/B:JCNS.0000037683.55688.7e PubMed DOI

Bazhenov M, Stopfer M. Forward and back: motifs of inhibition in olfactory processing. Neuron. 2010;67(3):357–8. doi: 10.1016/j.neuron.2010.07.023 PubMed DOI PMC

Bruno RM. Synchrony in sensation. Curr Opin Neurobiol. 2011;21(5):701–8. doi: 10.1016/j.conb.2011.06.003 PubMed DOI PMC

Dong H, Shao Z, Nerbonne JM, Burkhalter A. Differential depression of inhibitory synaptic responses in feedforward and feedback circuits between different areas of mouse visual cortex. J Comp Neurol. 2004;475(3):361–73. doi: 10.1002/cne.20164 PubMed DOI

Pouille F, Scanziani M. Enforcement of temporal fidelity in pyramidal cells by somatic feed-forward inhibition. Science. 2001;293(5532):1159–63. doi: 10.1126/science.1060342 PubMed DOI

Shao Z, Burkhalter A. Different balance of excitation and inhibition in forward and feedback circuits of rat visual cortex. Journal of Neuroscience. 1996;16(22):7353–65. doi: 10.1523/JNEUROSCI.16-22-07353.1996 PubMed DOI PMC

Silberberg G. Polysynaptic subcircuits in the neocortex: spatial and temporal diversity. Curr Opin Neurobiol. 2008;18(3):332–7. doi: 10.1016/j.conb.2008.08.009 PubMed DOI

Bazhenov M, Rulkov NF, Fellous JM, Timofeev I. Role of network dynamics in shaping spike timing reliability. Phys Rev E Stat Nonlin Soft Matter Phys. 2005;72(4 Pt 1):041903. doi: 10.1103/PhysRevE.72.041903 PubMed DOI

Rulkov NF. Modeling of spiking-bursting neural behavior using two-dimensional map. Phys Rev E Stat Nonlin Soft Matter Phys. 2002;65(4 Pt 1):041922. doi: 10.1103/PhysRevE.65.041922 PubMed DOI

Komarov M, Krishnan G, Chauvette S, Rulkov N, Timofeev I, Bazhenov M. New class of reduced computationally efficient neuronal models for large-scale simulations of brain dynamics. J Comput Neurosci. 2018;44(1):1–24. doi: 10.1007/s10827-017-0663-7 PubMed DOI

Python/NEURON code for simulating biophysically realistic thalamocortical dynamics during sleep