Weight statistics controls dynamics in recurrent neural networks

Journal article


Publication Details

Author(s): Krauß P, Schuster M, Dietrich V, Schilling A, Schulze H, Metzner C
Journal: PLoS ONE
Publication year: 2019
Volume: 14
Journal issue: 4
ISSN: 1932-6203


Abstract

Recurrent neural networks are complex non-linear systems, capable of ongoing activity in the absence of driving inputs. The dynamical properties of these systems, in particular their long-time attractor states, are determined on the microscopic level by the connection strengths w(ij) between the individual neurons. However, little is known to which extent network dynamics is tunable on a more coarse-grained level by the statistical features of the weight matrix. In this work, we investigate the dynamics of recurrent networks of Boltzmann neurons. In particular we study the impact of three statistical parameters: density (the fraction of non-zero connections), balance (the ratio of excitatory to inhibitory connections), and symmetry (the fraction of neuron pairs with w(ij)= w(ij)). By computing a 'phase diagram' of network dynamics, we find that balance is the essential control parameter: Its gradual increase from negative to positive values drives the system from oscillatory behavior into a chaotic regime, and eventually into stationary fixed points. Only directly at the border of the chaotic regime do the neural networks display rich but regular dynamics, thus enabling actual information processing. These results suggest that the brain, too, is fine-tuned to the 'edge of chaos' by assuring a proper balance between excitatory and inhibitory neural connections.


FAU Authors / FAU Editors

Krauß, Patrick Dr.
Hals-Nasen-Ohren-Klinik - Kopf- und Halschirurgie
Metzner, Claus PD Dr.
Naturwissenschaftliche Fakultät
Schilling, Achim Dr. rer. nat.
Hals-Nasen-Ohren-Klinik - Kopf- und Halschirurgie
Schulze, Holger Prof. Dr. rer. nat.
Professur für Experimentelle HNO-Heilkunde


How to cite

APA:
Krauß, P., Schuster, M., Dietrich, V., Schilling, A., Schulze, H., & Metzner, C. (2019). Weight statistics controls dynamics in recurrent neural networks. PLoS ONE, 14(4). https://dx.doi.org/10.1371/journal.pone.0214541

MLA:
Krauß, Patrick, et al. "Weight statistics controls dynamics in recurrent neural networks." PLoS ONE 14.4 (2019).

BibTeX: 

Last updated on 2019-29-04 at 09:38