Higher-order interactions underlie complex phenomena in systems such as biological and artificial neural networks, but their study is challenging due to the scarcity of tractable models. By leveraging a generalisation of the maximum entropy principle, we introduce curved neural networks as a class of models with a limited number of parameters that are particularly well-suited for studying higher-order phenomena. Through exact mean-field descriptions, we show that these curved neural networks implement a self-regulating annealing process that can accelerate memory retrieval, leading to explosive order-disorder phase transitions with multi-stability and hysteresis effects. Moreover, by analytically exploring their memory-retrieval capacity using the replica trick, we demonstrate that these networks can enhance memory capacity and robustness of retrieval over classical associative-memory networks. Overall, the proposed framework provides parsimonious models amenable to analytical study, revealing higher-order phenomena in complex networks.

BCAM Basque Center for Applied Mathematics Bilbao Spain

Center for Eudaimonia and Human Flourishing University of Oxford Oxford UK

Center for Human Nature Artificial Intelligence and Neuroscience Hokkaido University Sapporo Japan

Centre for Complexity Science Imperial College London London UK

Department of Brain Sciences and Centre for Complexity Science Imperial College London London UK

Graduate School of Informatics Kyoto University Kyoto Japan

IKERBASQUE Basque Foundation for Science Bilbao Spain

Principles of Intelligent Behavior in Biological and Social Systems Prague Czech Republic

Research Division Araya Inc Tokyo Japan

Sussex AI and Sussex Centre for Consciousness Science Department of Informatics University of Sussex Brighton UK

Zobrazit více v PubMed

Lambiotte, R., Rosvall, M. & Scholtes, I. From networks to optimal higher-order models of complex systems. PubMed PMC

Battiston, F. et al. The physics of higher-order interactions in complex systems.

Amari, S.-i, Nakahara, H., Wu, S. & Sakai, Y. Synchronous firing and higher-order interactions in neuron pool. PubMed

Kuehn, C. & Bick, C. A universal route to explosive phenomena. PubMed PMC

Shomali, S. R., Rasuli, S. N., Ahmadabadi, M. N. & Shimazaki, H. Uncovering hidden network architecture from spiking activities using an exact statistical input-output relation of neurons. PubMed PMC

Thibeault, V., Allard, A. & Desrosiers, P. The low-rank hypothesis of complex systems.

Angst, S., Dahmen, S. R., Hinrichsen, H., Hucht, A. & Magiera, M. P. Explosive ising.

D’Souza, R. M., Gómez-Gardenes, J., Nagler, J. & Arenas, A. Explosive phenomena in complex networks.

Iacopini, I., Petri, G., Barrat, A. & Latora, V. Simplicial models of social contagion. PubMed PMC

Millán, A. P., Torres, J. J. & Bianconi, G. Explosive higher-order Kuramoto dynamics on simplicial complexes. PubMed

Landry, N. W. & Restrepo, J. G. The effect of heterogeneity on hypergraph contagion models. PubMed PMC

Montani, F. et al. The impact of high-order interactions on the rate of synchronous discharge and information transmission in somatosensory cortex. PubMed

Tkačik, G. et al. Searching for collective behavior in a large network of sensory neurons. PubMed PMC

Ohiorhenuan, I. E. et al. Sparse coding and high-order correlations in fine-scale cortical networks. PubMed PMC

Shimazaki, H., Sadeghi, K., Ishikawa, T., Ikegaya, Y. & Toyoizumi, T. Simultaneous silence organizes structured higher-order interactions in neural populations. PubMed PMC

Tkačik, G. et al. The simplest maximum entropy model for collective behavior in a neural network.

Tkačik, G. et al. Thermodynamics and signatures of criticality in a network of neurons. PubMed PMC

Burns, T. F. & Fukai, T. Simplicial Hopfield networks. In:

Bybee, C. et al. Efficient optimization with higher-order Ising machines. PubMed PMC

Krotov, D. & Hopfield, J. J. Dense associative memory for pattern recognition.

Demircigil, M., Heusel, J., Löwe, M., Upgang, S. & Vermet, F. On a model of associative memory with huge storage capacity.

Agliari, E. et al. Dense Hebbian neural networks: a replica symmetric picture of unsupervised learning.

Lucibello, C. & Mézard, M. Exponential capacity of dense associative memories. PubMed

Krotov, D. A new frontier for Hopfield networks.

Ambrogioni, L. In search of dispersed memories: Generative diffusion models are associative memory networks. PubMed PMC

Ambrogioni, L. The statistical thermodynamics of generative diffusion models: Phase transitions, symmetry breaking, and critical instability. PubMed PMC

Bovier, A. & Niederhauser, B. The spin-glass phase-transition in the Hopfield model with p-spin interactions.

Agliari, E., Fachechi, A. & Marullo, C. Nonlinear PDEs approach to statistical mechanics of dense associative memories.

Amari, S.-i. Information geometry on hierarchy of probability distributions.

Skardal, P. S. & Arenas, A. Higher order interactions in complex networks of phase oscillators promote abrupt synchronization switching.

Ganmor, E., Segev, R. & Schneidman, E. Sparse low-order interaction network underlies a highly correlated and learnable neural population code. PubMed PMC

Barra, A., Beccaria, M. & Fachechi, A. A new mechanical approach to handle generalized Hopfield neural networks. PubMed

Agliari, E., Barra, A. & Notarnicola, M. The relativistic Hopfield network: rigorous results.

Agliari, E., Alemanno, F., Barra, A. & Fachechi, A. Generalized guerra’s interpolation schemes for dense associative neural networks. PubMed

Rodríguez-Domínguez, U. & Shimazaki, H. Alternating shrinking higher-order interactions for sparse neural population activity. Preprint at https://arxiv.org/abs/2308.13257 (2023).

Santos, S., Niculae, V., McNamee, D. & Martins, A. F. Hopfield-fenchel-young networks: a unified framework for associative memory retrieval. Preprint at https://arxiv.org/abs/2411.08590 (2024).

Hoover, B., Chau, D. H., Strobelt, H., Ram, P. & Krotov, D. Dense associative memory through the lens of random features.

Jaynes, E. T.

Cofré, R., Herzog, R., Corcoran, D. & Rosas, F. E. A comparison of the maximum entropy principle across biological spatial scales.

Jaynes, E. T. Information theory and statistical mechanics.

Tsallis, C., Mendes, R. & Plastino, A. R. The role of constraints within generalized nonextensive statistics.

Morales, P. A. & Rosas, F. E. Generalization of the maximum entropy principle for curved statistical manifolds.

Valverde-Albacete, F. & Peláez-Moreno, C. The case for shifting the Rényi entropy. PubMed PMC

Umarov, S., Tsallis, C. & Steinberg, S. On aq-central limit theorem consistent with nonextensive statistical mechanics.

Wong, T.-K. L. & Zhang, J. Tsallis and rényi deformations linked via a new

Guisande, N. & Montani, F. Rényi entropy-complexity causality space: a novel neurocomputational tool for detecting scale-free features in EEG/iEEG data. PubMed PMC

Jauregui, M., Zunino, L., Lenzi, E. K., Mendes, R. S. & Ribeiro, H. V. Characterization of time series via rényi complexity–entropy curves.

Wong, T.-K. L. Logarithmic divergences from optimal transport and rényi geometry.

Vigelis, R. F., De Andrade, L. H. & Cavalcante, C. C. Properties of a generalized divergence related to Tsallis generalized divergence.

Amari, S.-I.

Roudi, Y., Dunn, B. & Hertz, J. Multi-neuronal activity and functional connectivity in cell assemblies. PubMed

Montúfar, G. in

Nakano, K. Associatron-a model of associative memory.

Amari, S.-I. Learning patterns and pattern sequences by self-organizing nets of threshold elements.

Hopfield, J. J. Neural networks and physical systems with emergent collective computational abilities. PubMed PMC

Amit, D. J.

Coolen, A. C., Kühn, R. & Sollich, P.

Coolen, A. In

Coolen, A. In

Mattis, D. Solvable spin systems with random interactions.

Kochmański, M., Paszkiewicz, T. & Wolski, S. Curie–Weiss magnet—a simple model of phase transition.

Amit, D. J., Gutfreund, H. & Sompolinsky, H. Storing infinite numbers of patterns in a spin-glass model of neural networks. PubMed

Bovier, A., Gayrard, V. & Picco, P. Gibbs states of the Hopfield model with extensively many patterns.

Talagrand, M. Rigorous results for the Hopfield model with many patterns.

Shcherbina, M. & Tirozzi, B. The free energy of a class of Hopfield models.

Krizhevsky, A.

Fontanari, J. F. & Theumann, W. On the storage of correlated patterns in Hopfield’s model.

Agliari, E., Barra, A., De Antoni, A. & Galluzzi, A. Parallel retrieval of correlated patterns: from Hopfield networks to Boltzmann machines. PubMed

Sherrington, D. & Kirkpatrick, S. Solvable model of a spin-glass.