Encoding Strategies in Spiking Neural Networks

September 10, 2020

Reading time ~2 minutes

Having attended a workshop on Spiking Neural Networks (SNN) recently I noticed that there is no established set of knowledge about encoding strategies.

In SNN usually there is a continuous analog signal. In digital simulations this signal is discretized. For processing in SNN this signal must be encoded. There are three major strategies to do so.

Rate Code

When the information is transmitted via the firing rate, it must happen in the range between no firing and the maximum firing rate. A negative firing rate is not possible. When utilizing rate code and the values to be encoded can be positive and negative, there are two ways to encode this information. Two neurons per input dimension are needed to either set the ’positive’ or’ negative’ neuron to fire, depending on the sign of the encoded value. Another option is to add a bias value in the transcoding process so that negative values can be encoded in the range below this base level. The translation of an analog signal to a spiking firing rate happens via spike generators. Generators can generate sinusoidal spikes or Poisson distributed. Sinusoidal generators may create phase-locked signal propagation, while Poisson distributed spikes prevent this by evening out repeated correlations.

Place Cell Encoding

Also called labeled line in the cochlea. This is what we use in one-hot encoded vectors in ANNs. In this case the whole sensor space is covered in a one-to-one correspondence. Receptive fields are topological connected regions in the sensory space. Many sensors are connected to a cell.

There are parallels in convolutional neural networks. Each neuron in a CNN layer has a receptive field, with the size of the kernel. CNN layers are usually combined with a pooling layer. Pooling layer report the average or maximum activity to perform a size reduction. These resemble more the idea of a receptive field.

Time to First Spike

This is similar to the rate code, as the inverse of the firing rate. This encoding strategy is quite successful.

t_0 \approx f^{-1}

The benefit is that you have the earlier and less latency. Downside is that it is not robust as a single spike event encodes all the information. As soon as a spike is recorded in the output layer the processing can be stopped.