Spike-frequency adaptation contributes long short-term memory to networks of spiking neurons

Brains are able to integrate memory from the recent past into their online processing of information, seemingly without effort. This ability is critical for cognitive tasks such as speech understanding or operations on sequences of symbols such as numbers, letters or words. But it has remained unknown how networks of spiking neurons in the brain can achieve that. Facilitating synaptic connections in the prefrontal cortex may help, but their time constants are rather short, and do not explain flexible uses of working memory in brain areas where they are not present. We show that the presence of neurons with spike frequency adaptation makes a significant difference: Their inclusion in a network moves its performance for such computing tasks from a very low level close to the level of human performance. While artificial neural networks with special long short-term memory (LSTM) units had already reached such high performance levels, they lack biological plausibility. We find that neurons with spike-frequency adaptation (SFA) provide to brains a functional equivalent to LSTM units. We call these biologically plausible spiking recurrent networks with long-short term memory LSNNs. These LSNNs also provide interpretable insights into emergent structures and low dimensional representations in network activity for such cognitive tasks.