计算机工程与应用 ›› 2022, Vol. 58 ›› Issue (6): 80-87.DOI: 10.3778/j.issn.1002-8331.2103-0118

• 理论与研发 • 上一篇    下一篇

基于动量分数阶梯度的卷积神经网络优化方法

郭明霄,王宏伟,王佳,李昊哲,杨仕旗   

  1. 1.新疆大学 电气工程学院,乌鲁木齐 830047
    2.大连理工大学 控制科学与工程学院,辽宁 大连 116024
    3.大连医科大学 基础医学院,辽宁 大连 116041
  • 出版日期:2022-03-15 发布日期:2022-03-15

Convolutional Neural Network Optimization Method Based on Momentum Fractional Order Gradient Descent Algorithm

GUO Mingxiao, WANG Hongwei, WANG Jia, LI Haozhe, YANG Shiqi   

  1. 1.School of Electrical Engineering, Xinjiang University, Urumqi 830047, China
    2.School of Control Science and Engineering, Dalian University of Technology, Dalian, Liaoning 116024, China
    3.College of Basic Medical Sciences, Dalian Medical University, Dalian, Liaoning 116041, China
  • Online:2022-03-15 Published:2022-03-15

摘要: 针对采用传统梯度下降算法训练卷积神经网络收敛速度慢的问题,提出了动量分数阶梯度下降算法。介绍了分数阶微积分的定义,并依据问题描述,通过算法推导,将整数阶梯度下降算法中的动量思想应用到分数阶梯度下降算法中,设计出动量分数阶梯度下降算法。使用测试函数验证算法的收敛性,并分析不同分数阶阶次和动量项系数对算法收敛性的影响。在三个数据集上使用动量分数阶梯度下降算法与传统梯度下降算法、动量梯度下降算法作对比实验,实验数据表明,动量分数阶梯度下降算法可以在不同复杂程度的数据集上,在保证较高分类准确率的前提下,极大提高卷积神经网络的收敛速度,为训练卷积神经网络节省大量时间成本。

关键词: 卷积神经网络, 分数阶, 梯度下降, 动量

Abstract: Aiming at the slow convergence speed of convolutional neural network trained by traditional gradient descent algorithm, a momentum fractional order gradient descent algorithm is proposed. The definition of fractional order calculus is introduced, and according to the problem description, the momentum thought of integer order gradient descent algorithm is applied to the fractional order gradient descent algorithm through algorithm derivation, then the momentum fractional order gradient descent algorithm is designed. The convergence of the algorithm is verified by using the test function, and the influence of different fractional orders and momentum coefficients on the algorithm are analyzed. The momentum fractional order gradient descent algorithm is used to compare with the traditional gradient descent algorithm and momentum gradient descent algorithm on three datasets. The experimental data shows that the momentum fractional order gradient descent algorithm can greatly improve the convergence speed of convolutional neural network on datasets with different complexity levels, while ensuring high classification accuracy, and saving a lot of training time.卷积神经网络;分数阶;梯度下降;动量

Key words: convolutional neural network, fractional order, gradient descent, momentum