SNN Training Algorithm Based on Relationship Between Pulse Frequency and Input Current

doi:10.3778/j.issn.1002-8331.2106-0506

Abstract

Abstract: Spiking neural network（SNN） is driven by asynchronous events and supports massively parallel computing. It has great potential in improving the computational efficiency of synchronous analog neural networks. However, SNN is still facing the problem that it cannot be directly trained. For this reason, inspired by the research on the response mechanism of leaky integrate-and-fire（LIF） model, this paper proposes a new SNN training algorithm based on rate coding. Firstly, this paper models the spike rate of the LIF neuron through the simulation experiment, obtains the expression relationship between the spike rate and the input current, and takes the derivative as the gradient, which solves the non-differentiability of the discrete spike event during the SNN training process. Secondly, this paper uses the LIF neuron response mechanism to update the network parameters, which improves the learning efficiency. The experimental results on the MNIST and CIFAR10 datasets verify the effectiveness of this method. The classification accuracy reaches 99.53% and 89.46%. The recognition accuracy on CIFAR10 data is 4.22 percentage points higher than that of previous researchers and the learning efficiency has been doubled.

Key words: spiking neural network（SNN）, backpropagation algorithm, LIF neuron, spike rate, neuromorphic brain-like computing

摘要： 脉冲神经网络（spiking neural network，SNN）以异步事件驱动，支持大规模并行计算，在改善同步模拟神经网络的计算效率方面具有巨大潜力。然而，目前SNN仍然面临无法直接训练的难题，为此，受到神经科学领域关于LIF（leaky integrate-and-fire）神经元响应机制研究启发，提出了一种新的基于频率编码的SNN训练算法。通过仿真实验对LIF神经元发放脉冲频率进行建模，得到LIF神经元脉冲频率与输入电流之间显示表达关系，并将其导数作为梯度，解决了SNN训练过程中离散脉冲事件产生的不可微性问题，使得可利用BP算法对SNN进行训练。现有基于频率编码的方法采用时间信用分配机制进行参数更新，通常具有较差的学习效率，为此，采用LIF神经元响应机制更新网络参数，提高了学习效率。在MNIST和CIFAR10数据集上的实验结果验证了所提方法的有效性，分类精度分别达到了99.53%和89.46%，在CIFAR10数据上的识别精度相较于先前研究者提高了4.22个百分点，在学习效率方面相较于先前采用时间信用分配方法提高了一倍左右。

关键词: 脉冲神经网络（SNN）, 反向传播算法, LIF神经元, 脉冲频率, 神经形态类脑计算

LAN Haoxin, CHEN Yunhua. SNN Training Algorithm Based on Relationship Between Pulse Frequency and Input Current[J]. Computer Engineering and Applications, 2022, 58(10): 87-92.

兰浩鑫, 陈云华. 基于脉冲频率与输入电流关系的SNN训练算法[J]. 计算机工程与应用, 2022, 58(10): 87-92.

References

[1] DENG L，WU Y J，HU X，et al.Rethinking the performance comparison between SNNs and ANNs[J].Neural Networks，2019，121：294-307.
[2] BOHTE S M，KOK J N.Error-backpropagation in temporally encoded networks of spiking neurons[J].Neurocomputing，2002，48：17-37.
[3] GHOSH-DASTIDAR S，ADELI H.A new supervised learning algorithm for multiple spiking neural networks with application in epilepsy and seizure detection[J].Neural Networks：the Official Journal of the International Neural Network Society，2009，22（10）：1419-1431.
[4] MOSTAFA H.Supervised learning based on temporal coding in spiking neural networks[J].IEEE Transactions on Neural Networks and Learning Systems，2018，29（7）：3227-3235.
[5] LEE J H，DELBRUCK T，PFEIFFER M.Training deep spiking neural networks using backpropagation[J].Frontiers in Neuroscience，2016，508（10）：504-517.
[6] JIN Y，ZHANG W，LI P.Hybrid macro/micro level backpropagation for training deep spiking neural networks[J].arXiv：1805.07866，2018.
[7] WU Y J，DENG L，LI G Q，et al.Spatio-temporal backpropagation for training high-performance spiking neural networks[J].Frontiers in Neuroscience，2018，12：331.
[8] WU Y J，DENG L，LI G Q，et al.Direct training for spiking neural networks：faster，larger，better[C]//Proceedings of the AAAI Conference on Artificial Intelligence，2019，33：1311-1318.
[9] KASABOV N K.Time-space，spiking neural networks and brain-inspired artificial intelligence[M].Berlin，Heidelberg：Springer，2019：104-108.
[10] PIERRE Y.PyNN：a common interface for neuronal network simulators[J].Frontiers in Neuroinformatics，2009，2（11）：204-123.
[11] LIU Q，FURBER S.Noisy Softplus：a biology inspired activation function[C]//International Conference on Neural Information Processing.Cham：Springer，2016：405-412.
[12] LIU Q，CHEN Y，FURBER S.Noisy Softplus：an activation function that enables SNNs to be trained as ANNs[J].arXiv：1706.03609，2017.
[13] GIUGLIANO M，CAMERA G L，FUSI S，et al.The response of cortical neurons to in vivo-like input current：theory and experiment[J].Biological Cybernetics，2008，99（4）：303-318.
[14] GU P J，XIAO R，PAN G，et al.STCA：spatio-temporal credit assignment with delayed feedback in deep spiking neural networks[C]//28th International Joint Conference on Artificial Intelligence，2019：1366-1372.
[15] HAYKIN S，KOSKO B.Gradient based learning applied to document recognition[J].Intelligent Signal Processing，1998，86：2278-2324.
[16] KRIZHEVSKY A，NAIR V，HINTON G.The CIFAR-10 dataset[EB/OL].（2014）[2021-04-24].http：//www.cs.toronto.edu/kriz/cifar.html.
[17] ESSER S K，MEROLLA P A，ARTHUR J V，et al.Convolutional networks for fast，energy-efficient neuromorphic computing[J].Proceedings of the National Academy of Sciences，2016，113（41）：11441-11446.
[18] WU J，CHUA Y，ZHANG M，et al.Deep spiking neural network with spike count based learning rule[C]//2019 International Joint Conference on Neural Networks，2019：1-6.
[19] CHENG X，HAO Y Z，XU J M，et al.LISNN：improving spiking neural networks with lateral interactions for robust object recognition[C]//29th International Joint Conference on Artificial Intelligence，2020：1519-1525.
[20] DIEHL P U，NEIL D，BINAS J，et al.Fast-classifying，high-accuracy spiking deep networks through weight and threshold balancing[C]//2015 International Joint Conference on Neural Networks，2015：1-8.
[21] SENGUPTA A，YE Y，WANG R，et al.Going deeper in spiking neural networks：VGG and residual architectures[J].Frontiers in Neuroscience，2019，13：95.
[22] PANDA P，ROY K.Unsupervised regenerative learning of hierarchical features in spiking deep networks for object recognition[C]//2016 International Joint Conference on Neural Networks，2016：299-306.