MC-NAS:一种可视化模块贡献神经架构搜索方法

doi:10.3778/j.issn.1002-8331.2303-0046

摘要/Abstract

摘要： 现有的神经架构搜索方法无法直观地将网络模型与候选模块以及模型识别准确率之间的关系展示出来；同时很多NAS方法可扩展性差，无法将其搜索策略扩展至任意搜索空间。针对上述挑战，提出了一种可视化模块贡献神经架构搜索方法。提出了模块贡献这个概念，并通过分析贡献计算过程的窘境给出了任意搜索空间下的统一采样原则，利用统一的贡献度指导原则给出了不同搜索空间的贡献度计算策略。针对特定的约束条件通过动态网络规划算法生成神经网络体系结构。大量的实验结果表明该算法在任意搜索空间中的有效性。使用CIFAR-10、CIFAR-100和ImageNet16-120数据集在NAS-Bench-201基准测试上平均准确率达到了93.33%、71.07%、42.69%。

关键词: 神经架构搜索, 动态网络规划, 可视化模块贡献, 链式搜索空间, cell-based搜索空间

Abstract: The existing NAS methods can not directly show the relationship between network models and candidate modules and the accuracy of model recognition. At the same time, many NAS methods have poor scalability and cannot extend their search strategies to arbitrary search space. In response to the above challenges, this paper proposes a visual module contribution neural architecture search method. In this paper, the concept of module contribution is first proposed, and the unified sampling principle in arbitrary search space is given by analyzing the dilemma of the contribution calculation process. Finally, the neural network architecture is generated through a dynamic network programming algorithm for specific constraints. Extensive experimental results demonstrate the effectiveness of the proposed algorithm. Using the CIFAR?10, CIFAR?100, and ImageNet16?120 datasets, the average accuracy on the NAS-Bench-201 benchmark is 93.33%, 71.07%, and 42.69%, respectively.

Key words: neural architecture search, dynamic network programming, visual module contribution, chain-structured search space, cell-based search space

张睿, 李吉, 柴艳峰. MC-NAS:一种可视化模块贡献神经架构搜索方法[J]. 计算机工程与应用, 2024, 60(12): 118-128.

ZHANG Rui, LI Ji, CHAI Yanfeng. MC-NAS:Visual Module Contribution Neural Architecture Search Method[J]. Computer Engineering and Applications, 2024, 60(12): 118-128.

参考文献

[1] DONG Y N, LIU Q W, DU B, et al. Weighted feature fusion of convolutional neural network and graph attention network for hyperspectral image classification[J]. IEEE Transactions on Image Processing, 2022, 31: 1559-1572.
[2] HASSANZADEH T, ESSAM D, SARKER R. EvoDCNN: an evolutionary deep convolutional neural network for image classification[J]. Neurocomputing, 2022, 488: 271-283.
[3] CHANG Y L, TAN T H, LEE W H, et al. Consolidated convolutional neural network for hyperspectral image classification[J]. Remote Sensing, 2022, 14(7): 1571.
[4] ZHANG R, BAI X L, PAN L H, et al. Zero-small sample classification method with model structure self-optimization and its application in capability evaluation[J]. Applied Intelligence, 2022, 52: 5696-5717.
[5] HUANG G, LIU Z, VAN DER MAATEN L, et al. Densely connected convolutional networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017: 4700-4708.
[6] HOWARD A G, ZHU M, CHEN B, et al. Mobilenets: efficient convolutional neural networks for mobile vision applications[J]. arXiv:1704.04861, 2017.
[7] SZEGEDY C, VANHOUCKE V, IOFFE S, et al. Rethinking the inception architecture for computer vision[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016: 2818-2826.
[8] ZHANG Z, LIU S H, ZHANG Y, et al. RS-DARTS: a convolutional neural architecture search for remote sensing image scene classification[J]. Remote Sensing, 2021, 14(1): 141.
[9] LIU X B, ZHANG C C, CAI Z H, et al. Continuous particle swarm optimization-based deep learning architecture search for hyperspectral image classification[J]. Remote Sensing, 2021, 13(6): 1082.
[10] WU J J, KUANG H Y, LU Q H, et al. M-FasterSeg: an efficient semantic segmentation network based on neural architecture search[J]. Engineering Applications of Artificial Intelligence, 2022, 113: 104962.
[11] WANG Y, LI Y S, CHEN W, et al. DNAS: decoupling neural architecture search for high-resolution remote sensing image semantic segmentation[J]. Remote Sensing, 2022, 14(16): 3864.
[12] WU H W, ZHOU J T. IID-Net: image inpainting detection network via neural architecture search and attention[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2021, 32(3): 1172-1185.
[13] GUPTA A, SHETH P, XIE P T. Neural architecture search for pneumonia diagnosis from chest X-rays[J]. Scientific Reports, 2022, 12(1): 1-12.
[14] BAKER B, GUPTA O, NAIK N, et al. Designing neural network architectures using reinforcement learning[J]. arXiv:1611.02167, 2016.
[15] HU Y, WANG X, LI L, et al. Improving one-shot NAS with shrinking-and-expanding supernet[J]. Pattern Recognition, 2021, 118: 108025.
[16] ZHANG M, LI H, PAN S, et al. One-shot neural architecture search: maximising diversity to overcome catastrophic forgetting[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020, 43(9): 2921-2935.
[17] ZHANG J, LI D, WANG L, et al. One-shot neural architecture search by dynamically pruning supernet in hierarchical order[J]. International Journal of Neural Systems, 2021, 31(7): 2150029.
[18] YING C, KLEIN A, CHRISTIANSEN E, et al. Nas-bench-101: towards reproducible neural architecture search[C]//International Conference on Machine Learning, 2019: 7105-7114.
[19] DONG X, YANG Y. Nas-bench-201: extending the scope of reproducible neural architecture search[J]. arXiv:2001.00326, 2020.
[20] DONG X, LIU L, MUSIAL K, et al. Nats-bench: benchmarking nas algorithms for architecture topology and size[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2021, 44(7): 3634-3646.
[21] CAI H, GAN C, WANG T, et al. Once-for-all: train one network and specialize it for efficient deployment[J]. arXiv:1908.09791, 2019.
[22] LIU H, SIMONYAN K, YANG Y. Darts: differentiable architecture search[J]. arXiv:1806.09055, 2018.
[23] XU Y, XIE L, ZHANG X, et al. Pc-darts: partial channel connections for memory-efficient architecture search[J]. arXiv:1907.05737, 2019.
[24] HU S, XIE S, ZHENG H, et al. DSNAS: direct neural architecture search without parameter retraining[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020: 12084-12092.
[25] ZHENG X, JI R R, CHEN Y, et al. MIGO-NAS: towards fast and generalizable neural architecture search[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2021, 43(9): 2936-2952.
[26] CHEN W, GONG X, WANG Z. Neural architecture search on imagenet in four GPU hours: a theoretically inspired perspective[J]. arXiv:2102.11535, 2021.
[27] LI S, MAO Y X, ZHANG F C. et al. DLW-NAS: differentiable light-weight neural architecture search[J] Cognitive Computation, 2022, 15: 429-439.
[28] DING Y D, WU Y, HUANG C Y. et al. NAP: neural architecture search with pruning[J]. Neurocomputing, 2022, 477: 85-95.
[29] AKIMOTO Y, SHIRAKAWA S, YOSHINARI N, et al. Adaptive stochastic natural gradient method for one-shot neural architecture search[C]//International Conference on Machine Learning, 2019: 171-180.
[30] ZHENG X, JI R, TANG L, et al. Multinomial distribution learning for effective neural architecture search[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision, 2019: 1304-1313.
[31] ZHENG X, JI R, TANG L, et al. Dynamic distribution pruning for efficient network architecture search[J]. arXiv:1905.13543, 2019.
[32] SINHA N, CHEN K W. Evolving neural architecture using one shot model[C]//Proceedings of the Genetic and Evolutionary Computation Conference, 2021: 910-918.