计算机工程与应用 ›› 2022, Vol. 58 ›› Issue (11): 260-268.DOI: 10.3778/j.issn.1002-8331.2111-0098

• 工程与应用 • 上一篇    下一篇

用于智能垃圾分拣的注意力YOLOv4算法

李庆,龚远强,张玮,张洋,刘超,李军,韩丹,刘德峰,梅文豪,董雪   

  1. 1.上海交通大学 中英国际低碳学院,上海 200240 
    2.海安上海交通大学智能装备研究院,江苏 海安 226600
    3.中国天楹股份有限公司,江苏 海安 226600
  • 出版日期:2022-06-01 发布日期:2022-06-01

Attention YOLOv4 Algorithm for Intelligent Waste Sorting

LI Qing, GONG Yuanqiang, ZHANG Wei, ZHANG Yang, LIU Chao, LI Jun, HAN Dan, LIU Defeng, MEI Wenhao, DONG Xue   

  1. 1.China-UK Low Carbon College, Shanghai Jiao Tong University, Shanghai 200240, China
    2.Hai’an Intelligent Equipment Research Institute, Shanghai Jiao Tong University, Hai’an, Jiangsu 226600, China
    3.China Tian Ying Inc, Hai’an, Jiangsu 226600, China
  • Online:2022-06-01 Published:2022-06-01

摘要: 针对人工垃圾分拣效率低、工作环境恶劣且成本高的问题,提出了一套智能可回收垃圾分拣系统,该系统采用RGB图像作为视觉信息输入,通过目标检测算法获取垃圾在传送带上的位置坐标信息,并通过机械臂对垃圾进行分拣操作。可回收垃圾形态各异、种类繁多,为提高检测算法的泛化能力,建立了一个含36 572帧图片的可回收垃圾数据集,并基于此数据集上训练目标检测算法。基于YOLOv4提出了嵌入注意力机制的目标检测算法Attn-YOLOv4,经实验验证,Attn-YOLOv4算法的mAP比原始YOLOv4算法高0.16个百分点。在静态识别功能的基础上,提出基于多线程的目标跟踪算法实现了对运动垃圾的快速稳定跟踪,在20 mm误差范围内达到了0.945的精确度。此外,后处理模块对图像进行形态学处理并获取垃圾的世界坐标以及放置角度,供机械臂进行分拣操作。分别对目标检测和跟踪算法进行验证,在实际分拣流水线上验证并评估了该智能可回收垃圾分拣系统的可行性、精度及分拣的成功率。

关键词: 垃圾分拣, 单目视觉, 深度学习, 目标检测, 目标跟踪, YOLOv4, 注意力机制, 机械臂

Abstract: The disposal of recyclable waste in China primarily relies on manual sorting, which is inefficient, suffers from harsh working environment and high cost. In this study, an intelligent recyclable waste sorting system is proposed, which employs RGB image as visual information input. Then the class and coordinates of the moving waste on the conveyor belt are obtained through object detection algorithm. Finally, the recyclable waste is picked up by the robotic arm and delivered to different bins. As recyclable waste has various appearance, it collects 36 572 frames of waste pictures and establishes a recyclable waste dataset for model training. This is to improve the generalization ability of the object detection model. Based on YOLOv4, an attention-based detection algorithm, Attn-YOLOv4 is proposed, and the mAP of Attn-YOLOv4 algorithm is 0.16% higher than that of the original YOLOv4. Besides, a multithreaded object tracking algorithm is proposed, which achieves an accuracy of 0.945 within a tolerance of 20 mm. Furthermore, a post-processing module acquires the orientations of the waste, which is further converted to world coordinates and orientations for the robotic arm. In the end, the feasibility and practical effect of the intelligent recyclable waste sorting system are verified and evaluated on a continuous recycling production line.

Key words: waste sort, monocular vision, deep learning, object detection, object tracking, YOLOv4, attention mechanism, robotic arm