Grasp Pose Estimation Based on Multi-Scale Feature Fusion

doi:10.3778/j.issn.1002-8331.2012-0023

Abstract

Abstract: In order to improve the success rate of robot grasping, a multi-scale feature fusion method for robot grasping pose estimation is proposed. The method takes RGD information as input, uses ResNet-50 backbone network and integrates FPN（feature pyramid networks） to obtain multi-scale features as the input of grasping generation network to generate grasping candidate frame. The grasping direction coordinates are mapped as the classification task of grasping direction, and ROI Align is used to extract the region of interest, evaluate the grasping candidate box, and obtain the optimal grasping pose of the target. In order to verify the effectiveness of the proposed algorithm, the pose estimation experiment based on Cornell data set is carried out, and the accuracy of pose estimation reaches 96.9%. Based on the Inter RealSense D415 depth camera and UR5 manipulator, a real object platform is built. In the real scene, multiple grasping experiments are carried out on the diverse objects randomly placed in the real scene. The results show that the detection success rate of grasping target is 95.8%, and the success rate of robot grasping is 90.2%.

Key words: grasp pose estimation, RGD information, multi-scale features, grasp proposal network, ROI Align

摘要： 抓取目标多样性、位姿随机性严重制约了机器人抓取的任务适应性，为提高机器人抓取成功率，提出一种融合多尺度特征的机器人抓取位姿估计方法。该方法以RGD信息为输入，采用ResNet-50主干网络，融合FPN（feature pyramid networks）获得多尺度特征作为抓取生成网络的输入，以生成抓取候选框；并将抓取方向坐标映射为抓取方向的分类任务，使用ROI Align进行感兴趣区域提取，评估抓取候选框，获取目标的最优抓取位姿。为验证算法有效性，基于康奈尔抓取数据集开展了抓取位姿估计实验，仿真抓取位姿估计准确度达到96.9%。基于Inter RealSense D415深度相机和UR5机械臂搭建了实物平台，在真实场景下对位姿随机摆放的多样性目标物体进行多次抓取实验，结果显示抓取目标检测成功率为95.8%，机器人抓取成功率为90.2%。

关键词: 抓取位姿估计, RGD信息, 多尺度特征, 抓取建议网络, ROI Align

XIAO Xianpeng, HU Li, ZHANG Jing, LI Shuchun, ZHANG Hua. Grasp Pose Estimation Based on Multi-Scale Feature Fusion[J]. Computer Engineering and Applications, 2022, 58(10): 172-177.

肖贤鹏, 胡莉, 张静, 李树春, 张华. 基于多尺度特征融合的抓取位姿估计[J]. 计算机工程与应用, 2022, 58(10): 172-177.

References

[1] KOPICKI M，DETRY R，ADJIGBLE M，et al.One-shot learning and generation of dexterous grasps for novel objects[J].The International Journal of Robotics Research，2016，35（8）：959-976.
[2] MORRISON D，CORKE P，LEITNER J.Learning robust，real-time，reactive robotic grasping[J].The International Journal of Robotics Research，2020，39（2/3）：183-201.
[3] ANTANAS L，MORENO P，NEUMANN M，et al.Semantic and geometric reasoning for robotic grasping：a probabilistic logic approach[J].Autonomous Robots，2019，43（6）：1393-1418.
[4] KRIZHEVSKY A，SUTSKEVER I，HINTON G E.ImageNet classification with deep convolutional neural networks[J].Communications of the ACM，2017，60（6）：84-90.
[5] 黄家才，舒奇，朱晓春，等.基于迁移学习的机器人视觉识别与分拣策略[J].计算机工程与应用，2019，55（8）：232-237.
HUANG J C，SHU Q，ZHU X C，et al.Visual recognition and sorting strategy of robot based on transfer learning[J].Computer Engineering and Applications，2019，55（8）：232-237.
[6] 黄怡蒙，易阳.融合深度学习的机器人目标检测与定位[J].计算机工程与应用，2020，56（24）：181-187.
HUANG Y M，YI Y.Target detection and location of robots integrated with deep learning[J].Computer Engineering and Applications，2020，56（24）：181-187.
[7] YAN Q，CAI J，MA Y，et al.Robust learning control for robot manipulators with random initial errors and iteration-varying reference trajectories[J].IEEE Access，2019，7：32628-32643.
[8] 杨小琴，朱玉全.基于物联网的机器人运行路径感知规划仿真[J].计算机仿真，2020，37（8）：286-290.
YANG X Q，ZHU Y Q.Simulation of robot operation path perception planning based on Internet of things[J].Computer Simulation，2020，37（8）：286-290.
[9] REDMON J，ANGELOVA A.Real-time grasp detection using convolutional neural networks[C]//2015 IEEE International Conference on Robotics and Automation，2015：1316-1322.
[10] KUMRA S，KANAN C.Robotic grasp detection using deep convolutional neural networks[C]//2017 IEEE/RSJ International Conference on Intelligent Robots and Systems，2017：769-776.
[11] GUO D，SUN F，LIU H，et al.A hybrid deep architecture for robotic grasp detection[C]//2017 IEEE International Conference on Robotics and Automation，2017：1609-1614.
[12] CHU F J，XU R，VELA P A.Real-world multiobject，multigrasp detection[J].IEEE Robotics and Automation Letters，2018，3（4）：3355-3362.
[13] YUAN J，XIONG H C，XIAO Y，et al.Gated CNN：integrating multi-scale feature layers for object detection[J].Pattern Recognition，2019，105：107131.
[14] DEPIERRE A，DELLANDRéA E，CHEN L.Jacquard：a large scale dataset for robotic grasp detection[C]//2018 IEEE/RSJ International Conference on Intelligent Robots and Systems，2018：3511-3516.
[15] 赵丽萍，袁霄，祝承，等.面向图像分类的残差网络进展研究[J].计算机工程与应用，2020，56（20）：9-19.
ZHAO L P，YUAN X，ZHU C，et al.Advances in residuals network for image classification[J].Computer Engineering and Applications，2020，56（20）：9-19.