Attention YOLOv4 Algorithm for Intelligent Waste Sorting

doi:10.3778/j.issn.1002-8331.2111-0098

Abstract

Abstract: The disposal of recyclable waste in China primarily relies on manual sorting, which is inefficient, suffers from harsh working environment and high cost. In this study, an intelligent recyclable waste sorting system is proposed, which employs RGB image as visual information input. Then the class and coordinates of the moving waste on the conveyor belt are obtained through object detection algorithm. Finally, the recyclable waste is picked up by the robotic arm and delivered to different bins. As recyclable waste has various appearance, it collects 36 572 frames of waste pictures and establishes a recyclable waste dataset for model training. This is to improve the generalization ability of the object detection model. Based on YOLOv4, an attention-based detection algorithm, Attn-YOLOv4 is proposed, and the mAP of Attn-YOLOv4 algorithm is 0.16% higher than that of the original YOLOv4. Besides, a multithreaded object tracking algorithm is proposed, which achieves an accuracy of 0.945 within a tolerance of 20 mm. Furthermore, a post-processing module acquires the orientations of the waste, which is further converted to world coordinates and orientations for the robotic arm. In the end, the feasibility and practical effect of the intelligent recyclable waste sorting system are verified and evaluated on a continuous recycling production line.

Key words: waste sort, monocular vision, deep learning, object detection, object tracking, YOLOv4, attention mechanism, robotic arm

摘要： 针对人工垃圾分拣效率低、工作环境恶劣且成本高的问题，提出了一套智能可回收垃圾分拣系统，该系统采用RGB图像作为视觉信息输入，通过目标检测算法获取垃圾在传送带上的位置坐标信息，并通过机械臂对垃圾进行分拣操作。可回收垃圾形态各异、种类繁多，为提高检测算法的泛化能力，建立了一个含36 572帧图片的可回收垃圾数据集，并基于此数据集上训练目标检测算法。基于YOLOv4提出了嵌入注意力机制的目标检测算法Attn-YOLOv4，经实验验证，Attn-YOLOv4算法的mAP比原始YOLOv4算法高0.16个百分点。在静态识别功能的基础上，提出基于多线程的目标跟踪算法实现了对运动垃圾的快速稳定跟踪，在20 mm误差范围内达到了0.945的精确度。此外，后处理模块对图像进行形态学处理并获取垃圾的世界坐标以及放置角度，供机械臂进行分拣操作。分别对目标检测和跟踪算法进行验证，在实际分拣流水线上验证并评估了该智能可回收垃圾分拣系统的可行性、精度及分拣的成功率。

关键词: 垃圾分拣, 单目视觉, 深度学习, 目标检测, 目标跟踪, YOLOv4, 注意力机制, 机械臂

LI Qing, GONG Yuanqiang, ZHANG Wei, ZHANG Yang, LIU Chao, LI Jun, HAN Dan, LIU Defeng, MEI Wenhao, DONG Xue. Attention YOLOv4 Algorithm for Intelligent Waste Sorting[J]. Computer Engineering and Applications, 2022, 58(11): 260-268.

李庆, 龚远强, 张玮, 张洋, 刘超, 李军, 韩丹, 刘德峰, 梅文豪, 董雪. 用于智能垃圾分拣的注意力YOLOv4算法[J]. 计算机工程与应用, 2022, 58(11): 260-268.

References

[1] LIU W，ANGUELOV D，ERHAN D，et al.SSD：single shot multibox detector[C]//European Conference on Computer Vision.Cham：Springer，2016：21-37.
[2] REDMON J，DIVVALA S，GIRSHICK R，et al.You only look once：unified，real-time object detection[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2016：779-788.
[3] LIN T Y，GOYAL P，GIRSHICK R，et al.Focal loss for dense object detection[C]//Proceedings of the IEEE International Conference on Computer Vision，2017：2980-2988.
[4] BOCHKOVSKIY A，WANG C Y，LIAO H Y M.Yolov4：optimal speed and accuracy of object detection[J].arXiv：2004.10934，2020.
[5] MIKAMI K，CHEN Y，NAKAZAWA J，et al.DeepCounter：using deep learning to count garbage bags[C]//2018 IEEE 24th International Conference on Embedded and Real-Time Computing Systems and Applications（RTCSA），2018：1-10.
[6] AWE O，MENGISTU R，SREEDHAR V.Smart trash net：waste localization and classification[R].Stanford：Stanford University，2017.
[7] YANG M，THUNG G.Classification of trash for recyclability status[R].CS229 Project Report，2016.
[8] DONOVAN J.Auto-trash sorts garbage automatically at the techcrunch disrupt hackathon[R].2016.
[9] ZHANG P，ZHAO Q，GAO J，et al.Urban street cleanliness assessment using mobile edge computing and deep learning[J].IEEE Access，2019，7：63550-63563.
[10] LIANG S，GU Y.A deep convolutional neural network to simultaneously localize and recognize waste types in images[J].Waste Management，2021，126：247-257.
[11] 袁建野，南新元，李成荣，等.注意力机制下的轻量级垃圾分类网络[J/OL].计算机工程：1-9[2021-09-29].https：//doi.org/10.19678/j.issn.1000-3428.0058108.
YUAN J Y，NAN X Y，LI C R，et al.Attention under the mechanism of the lightweight trash classification networks[J/OL].Computer Engineering：1-9[2021-09-29].https：//doi.org/10.19678/j.issn.1000-3428.0058108.
[12] MA W，WANG X，YU J.A lightweight feature fusion single shot multibox detector for garbage detection[J].IEEE Access，2020，8：188577-188586.
[13] KUMAR S，YADAV D，GUPTA H，et al.A novel YOLOv3 algorithm-based deep learning approach for waste segregation：towards smart waste management[J].Electronics，2021，10（1）：14.
[14] 张志飞.面向城市固废的工业机器人分拣系统的研究与实现[D].杭州：浙江工业大学，2020.
ZHANG Z F.Research and implementation of industrial robot sorting system for urban solid waste[D].Hangzhou：Zhejiang University of Technology，2020.
[15] 周滢慜.基于机器视觉的生活垃圾智能分拣系统的设计与实现[D].哈尔滨：哈尔滨工业大学，2018.
ZHOU Y M.Design and implementation of intelligent garbage sorting system based on machine vision[D].Harbin：Harbin Institute of Technology，2018.
[16] BEWLEY A，GE Z，OTT L，et al.Simple online and realtime tracking[C]//2016 IEEE International Conference on Image Processing（ICIP），2016：3464-3468.
[17] WOJKE N，BEWLEY A，PAULUS D.Simple online and realtime tracking with a deep association metric[C]//2017 IEEE International Conference on Image Processing（ICIP），2017：3645-3649.
[18] BOLME D S，BEVERIDGE J R，DRAPER B A，et al.Visual object tracking using adaptive correlation filters[C]//2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition，2010：2544-2550.
[19] HENRIQUES J F，CASEIRO R，MARTINS P，et al.High-speed tracking with kernelized correlation filters[J].IEEE Transactions on Pattern Analysis and Machine Intelligence，2014，37（3）：583-596.
[20] DANELLJAN M，SHAHBAZ KHAN F，FELSBERG M，et al.Adaptive color attributes for real-time visual tracking[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2014：1090-1097.
[21] HU J，SHEN L，SUN G.Squeeze-and-excitation networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2018：7132-7141.
[22] WOO S，PARK J，LEE J Y，et al.Cbam：convolutional block attention module[C]//Proceedings of the European Conference on Computer Vision（ECCV），2018：3-19.
[23] 刘毅，刘传菊.应用于实时控制系统的线程池调度策略设计[J].计算机工程与应用，2010，46（32）：71-73.
LIU Y，LIU C J.Scheduling strategy design of thread pool for real-time control system[J].Computer Engineering and Applications，2010，46（32）：71-73.
[24] KU Y，YANG J，FANG H，et al.Deep learning of grasping detection for a robot used in sorting construction and demolition waste[J].Journal of Material Cycles and Waste Management，2021，23（1）：84-95.
[25] BABENKO B，YANG M H，BELONGIE S.Robust object tracking with online multiple instance learning[J].IEEE Transactions on Pattern Analysis and Machine Intelligence，2010，33（8）：1619-1632.
[26] WU Y，LIM J，YANG M H.Online object tracking：a benchmark[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2013：2411-2418.
[27] BOLME D S，BEVERIDGE J R，DRAPER B A，et al.Visual object tracking using adaptive correlation filters[C]//2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition，2010：2544-2550.