Improved Cherry Detection Method at Night Based on YOLOv7: YOLOv7-Cherry

doi:10.3778/j.issn.1002-8331.2306-0344

Abstract

Abstract: To solve the problem that the cherry detection algorithm can not recognize the maturity of cherries in the night environment, an improved YOLOv7 algorithm: YOLOv7-Cherry is proposed. An image preprocessing method is used to combine the nighttime cherry image with the daytime cherry image in the same position to preserve the high spatial resolution information of the nighttime cherry image and enhance its spectral resolution. In YOLOv7-Cherry, the CBAM attention is firstly inserted into the backbone, and the attention mechanism is used to strengthen the representational ability of the neural network, emphasize important features, ignore secondary features, and enhance the extraction of cherry target features. Secondly, the recognition of small cherries are enhanced in images by the target detection algorithm, the small target detection layer is added. Then the initial detection box size of the original network is improved. Finally, to reduce the loss of cherry targets caused by the occlusion, the Soft-NMS method is used for the redundancy removal of the detection box. The experimental results demonstrate that YOLOv7-Cherry can significantly detect mature and immature cherries in night conditions. Compared with the YOLOv3, Faster-RCNN, YOLOv4, YOLOv5 and original YOLOv7 models, mAP of YOLOv7-Cherry model increased by 26.88, 25.05, 22.51, 17.11 and 7.66 percentage points, among which the precision, recall, mAP and F1 are 93.9%, 94.7%, 97.4% and 94.3%.

Key words: image fusion, YOLOv7, object detection, small target, cherry recognition at night

摘要： 针对樱桃检测算法无法对夜晚环境下的樱桃进行成熟度识别的问题，提出一种改进的YOLOv7算法：YOLOv7-Cherry。使用一种将夜间樱桃图像和白天相同位置的樱桃图像相融合的图像预处理方法，保留夜间樱桃图像高空间分辨信息的同时加强其光谱分辨率。在YOLOv7-Cherry中，将CBAM注意力机制插入到骨干网络中，利用注意力机制强化神经网络的表征能力，强调重要特征，忽略次要特征，加强对樱桃目标特征的提取；为了加强目标检测算法对图像中小樱桃的识别，增加小目标检测层；改进了原始网络的初始检测框大小；为了减少遮挡对樱桃目标造成的损失，对检测框使用了Soft-NMS方法进行冗余去除。实验结果表明，YOLOv7-Cherry可以有效地识别出夜晚环境下的成熟樱桃和未成熟樱桃, 与YOLOv3、Faster-RCNN、YOLOv4、YOLOv5和原YOLOv7相比，YOLOv7-Cherry的mAP提高了26.88、25.05、22.51、17.11和7.66个百分点，其中，识别精度、召回率、mAP和F1为93.9%、94.7%、97.4%、94.3%。

关键词: 图像融合, YOLOv7, 目标检测, 小目标, 夜间樱桃识别

GAI Rongli, KONG Xiangzhou, QIN Shan, WEI Kai. Improved Cherry Detection Method at Night Based on YOLOv7: YOLOv7-Cherry[J]. Computer Engineering and Applications, 2024, 60(21): 315-323.

盖荣丽, 孔祥宙, 秦山, 魏凯. 基于YOLOv7改进的夜间樱桃检测方法：YOLOv7-Cherry[J]. 计算机工程与应用, 2024, 60(21): 315-323.

References

[1] 赵悦菡, 侯召华, 纪海鹏, 等. 樱桃生理变化及保鲜机理研究进展[J]. 食品研究与开发, 2021, 42(23): 197-203.
ZHAO Y H, HOU Z H, JI H P, et al. Advances in research on physiological changes and fresh-keeping mechanism of cherries[J]. Food Research and Development, 2021, 42(23): 197-203.
[2] REN S, HE K, GIRSHICK R, et al. Faster R-CNN: towards real-time object detection with region proposal networks[C]//Advances in Neural Information Processing Systems, 2015.
[3] REDMON J, DIVVALA S, GIRSHICK R, et al. You only look once: unified, real-time object detection[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016: 779-788.
[4] KAMILARIS A, PRENAFETA-BOLDú F X. Deep learning in agriculture: a survey[J]. Computers and Electronics in Agriculture, 2018, 147: 70-90.
[5] GAI R L, CHEN N, YUAN H. A detection algorithm for cherry fruits based on the improved YOLO?v4 model[J]. Neural Computing and Applications, 2021, 35: 13895-13906.
[6] ZHENG T, JIANG M, LI Y, et al. Research on tomato detection in natural environment based on RC-YOLOv4[J]. Computers and Electronics in Agriculture, 2022, 198: 07029.
[7] 赵元龙, 单玉刚, 袁杰. 改进YOLOv7与DeepSORT的佩戴口罩行人跟踪[J]. 计算机工程与应用, 2023, 59(6): 221-230.
ZHAO Y L, SHAN Y G, YUAN J. Wearing mask pedestrian tracking based on improved YOLOv7 and DeepSORT[J]. Computer Engineering and Applications, 2023, 59(6): 221-230.
[8] 王文杰, 贡亮, 汪韬, 等. 基于多源图像融合的自然环境下番茄果实识别[J]. 农业机械学报, 2021, 52(9): 156-164.
WANG W J, GONG L, WANG T, et al. Tomato fruit recognition based on multi-source fusion image segmentation algorithm in open environment[J]. Transactions of the Chinese Society for Agricultural Machinery, 2021, 52(9): 156-164.
[9] ZHOU Z M, WU Z J, WANG J, et al. Panchromatic and multi-spectral image fusion using IHS and variational models[C]//Proceedings of the 2012 5th International Congress on Image and Signal Processing (CISP 2012), Chongqing, China, 2012: 1077-1080.
[10] GHAHREMANI M, GHASSEMIAN H. Nonlinear IHS: a promising method for pan-sharpening[J]. IEEE Geoscience and Remote Sensing Letters, 2016, 13(11): 1606-1610.
[11] 王欢. 基于小波变换的多传感器遥感图像融合算法研究[D]. 湘潭: 湘潭大学, 2009.
WANG H. Research on multi-sensor remote sensing image fusion based on wavelet transform[D]. Xiangtan: Xiangtan University, 2009.
[12] 李章维, 胡安顺, 王晓飞. 基于视觉的目标检测方法综述[J]. 计算机工程与应用, 2020, 56(8): 1-9.
LI Z W, HU A S, WANG X F. Survey of vision based object detection methods[J]. Computer Engineering and Applications, 2020, 56(8): 1-9.
[13] YAN B, FAN P, LEI X, et al. A real-time apple targets detection method for picking robot based on improved YOLOv5[J]. Remote Sensing, 2021, 13(9): 1619.
[14] WU D, JIANG S, ZHAO E, et al. Detection of camellia oleifera fruit in complex scenes by using YOLOv7 and data augmentation[J]. Applied Sciences, 2022, 12(22): 11318.
[15] XIA Y, NGUYEN M, YAN W Q. A real-time kiwifruit detection based on improved YOLOv7[C]//Proceedings of the 37th International Conference on Image and Vision Computing (IVCNZ 2022), Auckland, New Zealand, November 24-25, 2022. Cham: Springer Nature Switzerland, 2023: 48-61.
[16] WOO S, PARK J, LEE J Y, et al. CBAM: convolutional block attention module[C]//Proceedings of the 15th European Conference on Computer Vision (ECCV 2018), Munich, Germany, September 8-14, 2018: 3-19.
[17] 邓宁. 基于深度学习的注意力机制的算法改进研究[D]. 重庆: 西南大学, 2022.
DENG N. Research on algorithm improvement of attention mechanism based on deep learning[D]. Chongqing: Southwest University, 2022.
[18] LI Z, JIANG X, SHUAI L, et al. A real-time detection algorithm for sweet cherry fruit maturity based on YOLOX in the natural environment[J]. Agronomy, 2022, 12(10): 2482.
[19] 赵辉, 乔艳军, 王红君, 等. 基于改进YOLOv3的果园复杂环境下苹果果实识别[J]. 农业工程学报, 2021, 37(16): 127-135.
ZHAO H, QIAO Y J, WANG H J, et al. Apple fruit recognition in complex orchard environment based on improved YOLOv3[J]. Transactions of the Chinese Society of Agricultural Engineering (Transactions of the CSAE), 2021, 37(16): 127-135.
[20] BODLA N, SINGH B, CHELLAPPA R, et al. Soft-NMS—improving object detection with one line of code[C]//Proceedings of the IEEE International Conference on Computer Vision, 2017: 5561-5569.
[21] 于波, 冯伟. 改进SSD算法及其在口罩佩戴检测中的应用[J]. 计算机仿真, 2022, 39(5): 488-493.
YU B, FENG W. Improved SSD algorithm and its application in mask wear detection[J]. Computer Simulation, 2022, 39(5): 488-493.
[22] ZHENG Z, WANG P, LIU W, et al. Distance-IoU loss: faster and better learning for bounding box regression[C]//Proceedings of the AAAI Conference on Artificial Intelligence, 2020: 12993-13000.
[23] 黄斌. 基于深度学习的视频人物识别算法研究[D]. 北京: 北京邮电大学, 2021.
HUANG B. Research on video character recognition algorithm based on deep learning[D]. Beijing: Beijing University of Posts and Telecommunications, 2021.
[24] 南晓虎, 丁雷. 深度学习的典型目标检测算法综述[J]. 计算机应用研究, 2020, 37(S2): 15-21.
NAN X H, DING L. Review of typical target detection algorithms for deep learning[J]. Application Research of Computers, 2020, 37(S2): 15-21.
[25] REDMON J, FARHADI A. YOLOv3: an incremental improvement[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018: 2311-2314.
[26] BOCHKOVSKIY A, WANG C Y, LIAO H Y M. YOLOv4: optimal speed and accuracy of object detection[J]. arXiv:2004.10934, 2020.
[27] CHEN Z Y, WU R H, LIN Y Y, et al. Plant disease recognition model based on improved YOLOv5[J]. Agronomy, 2022, 2(2): 365.
[28] WANG C Y, BOCHKOVSKIY A, LIAO H Y M. YOLOv7: trainable bag-of-freebies sets new state-of-the-art for real-time object detectors[J]. arXiv:2207.02696, 2022.
[29] 宋怀波, 尚钰莹, 何东健. 果实目标深度学习识别技术研究进展[J]. 农业机械学报, 2023, 54(1): 1-19.
SONG H B, SHANG Y Y, HE D J. Review on deep learning technology for fruit target recognition[J]. Transactions of the Chinese Society for Agricultural Machinery, 2023, 54(1): 1-19.