复杂环境中的多尺度目标抓取检测研究

doi:10.3778/j.issn.1002-8331.2503-0110

摘要/Abstract

摘要： 在复杂环境中的物体抓取姿态检测是智能机器人实现自主操作的关键技能之一。GSNet在六自由度抓取学习方面取得了良好的效果，但其方法在采样和学习过程中忽略了不同尺度特征对抓取位姿估计的影响，导致对不同大小物体的抓取精度存在一定局限性。提出一个多尺度特征融合模块，该模块采用三种采样策略来获取多尺寸物体特征，并作为新的种子点提取方法，增强抓取检测网络对不同尺寸物体的适应能力。引入多尺度并行卷积结构优化骨干网络，增强网络对物体几何特征的感知能力。在GraspNet-1Billion数据集上进行抓取姿态准确度评估，结果表明改进后的方法取得了较好的结果；同时进一步评估了对于多尺度物体场景中的抓取姿态估计能力，实验结果在估计中、大尺寸物体抓取姿态准确度中表现良好，相较于其他主流方法最高有16.83个百分点的提升，并在真实环境实验中验证了方法的有效性。

关键词: 六自由度抓取, 多尺度特征, 深度学习, 点云

Abstract: Object grasping posture detection in complex environments is a fundamental capability for intelligent robots to achieve autonomous operation. GSNet has demonstrated promising performance in six-degree-of-freedom grasping learning. However, its approach overlooks the impact of multi-scale features on grasp position estimation during sampling and learning, leading to limitations in grasping accuracy for objects of different sizes. To address this issue, a multi-scale feature fusion module is proposed, which uses three sampling strategies to obtain multi-size object features and serves as a new seed point extraction method to improve the adaptability of the grasping detection network to objects of different sizes. Additionally, a multi-scale parallel convolutional structure is introduced to optimize the backbone network, improving its ability to perceive the geometric characteristics of objects. Finally, the grasp pose estimation accuracy is evaluated on the GraspNet-1Billion dataset. The results demonstrate that the proposed method achieves superior performance compared to baseline approaches. Additional experiments in multi-scale object scenarios demonstrated that the method performs well in grasp pose estimation for medium and large-sized objects, achieving improvements of up to 16.83 percengtage pionts over state-of-the-art methods. The effectiveness of the approach is further validated in real-world robotic experiments.

Key words: six-degree-of-freedom grasping, multi-scale features, deep learning, point cloud

罗鑫, 臧强, 张永宏, 董天天, 柏宗春. 复杂环境中的多尺度目标抓取检测研究[J]. 计算机工程与应用, 2025, 61(24): 273-280.

LUO Xin, ZANG Qiang, ZHANG Yonghong, DONG Tiantian, BAI Zongchun. Research on Multi-Scale Target Grasping Detection in Complex Environments[J]. Computer Engineering and Applications, 2025, 61(24): 273-280.

参考文献

[1] CORRELL N, BEKRIS K E, BERENSON D, et al. Analysis and observations from the first Amazon picking challenge[J]. IEEE Transactions on Automation Science and Engineering, 2018, 15(1): 172-188.
[2] SUáREZ-RUIZ F, ZHOU X, PHAM Q C. Can robots assemble an IKEA chair?[J]. Science Robotics, 2018, 3(17): eaat6385.
[3] EDMONDS M, GAO F, XIE X, et al. Feeling the force: integrating force and pose for fluent discovery through imitation learning to open medicine bottles[C]//Proceedings of the 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems. Piscataway: IEEE, 2017: 3530-3537.
[4] MILLER A T, KNOOP S, CHRISTENSEN H I, et al. Automatic grasp planning using shape primitives[C]//Proceedings of the 2003 IEEE International Conference on Robotics and Automation. Piscataway: IEEE, 2003: 1824-1829.
[5] MORRISON D, LEITNER J, CORKE P. Closing the loop for robotic grasping: a real-time, generative grasp synthesis approach[C]//Proceedings of the Robotics: Science and Systems XIV. Robotics: Science and Systems Foundation, 2018.
[6] 闫明, 陶大鹏, 普园媛. 面向工业零件分拣系统的低纹理目标检测[J]. 中国图象图形学报, 2022, 27(8): 2418-2429.
YAN M, TAO D P, PU Y Y. Texture-less object detection method for industrial components picking system[J]. Journal of Image and Graphics, 2022, 27(8): 2418-2429.
[7] XU R N, CHU F J, VELA P A. GKNet: : grasp keypoint network for grasp candidates detection[J]. International Journal of Robotics Research, 2022, 41(4): 361-389.
[8] MAHLER J, MATL M, SATISH V, et al. Learning ambidextrous robot grasping policies[J]. Science Robotics, 2019, 4(26): eaau4984.
[9] TEN PAS A, GUALTIERI M, SAENKO K, et al. Grasp pose detection in point clouds[J]. International Journal of Robotics Research, 2017, 36(13/14): 1455-1473.
[10] 孙国栋, 贾俊杰, 李明晶, 等. 杂乱场景下小物体抓取检测研究[J]. 中国图象图形学报, 2024, 29(2): 468-477.
SUN G D, JIA J J, LI M J, et al. Small object grasping detection in cluttered scenes[J]. Journal of Image and Graphics, 2024, 29(2): 468-477.
[11] LIANG H Z, MA X J, LI S, et al. PointNetGPD: detecting grasp configurations from point sets[C]//Proceedings of the 2019 International Conference on Robotics and Automation. Piscataway: IEEE, 2019: 3629-3635.
[12] FISCHINGER D, WEISS A, VINCZE M. Learning grasps with topographic features[J]. International Journal of Robotics Research, 2015, 34(9): 1167-1194.
[13] MOUSAVIAN A, EPPNER C, FOX D. 6-DOF GraspNet: variational grasp generation for object manipulation[C]//Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision. Piscataway: IEEE, 2019: 2901-2910.
[14] 徐衍, 林云汉, 闵华松. 杂乱场景中多尺度注意力特征融合抓取检测网络[J]. 计算机系统应用, 2024, 33(5): 76-84.
XU Y, LIN Y H, MIN H S. Grasping detection network of multi-scale attention feature fusion in cluttered scenes[J]. Computer Systems and Applications, 2024, 33(5): 76-84.
[15] LI Y M, KONG T, CHU R H, et al. Simultaneous semantic and collision learning for 6-DoF grasp pose estimation[C]//Proceedings of the 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems. Piscataway: IEEE, 2021: 3571-3578.
[16] QIN Y Z, CHEN R, ZHU H, et al. S4G: amodal single-view single-shot SE(3) grasp detection in cluttered scenes[J]. arXiv: 1910.14218, 2019.
[17] BREYER M, CHUNG J J, OTT L, et al. Volumetric grasping network: real-time 6 DOF grasp detection in clutter[C]//Proceedings of the Conference on Robot Learning, 2021.
[18] NI P Y, ZHANG W G, ZHU X X, et al. PointNet++ grasping: learning an end-to-end spatial grasp generation algorithm from sparse point clouds[C]//Proceedings of the 2020 IEEE International Conference on Robotics and Automation. Piscataway: IEEE, 2020: 3619-3625.
[19] QI C R, YI L, SU H, et al. PointNet++: deep hierarchical feature learning on point sets in a metric space[C]//Proceedings of the 31st International Conference on Neural Information Processing Systems. New York: ACM, 2017: 5105-5114.
[20] SUNDERMEYER M, MOUSAVIAN A, TRIEBEL R, et al. Contact-GraspNet: efficient 6-DoF grasp generation in cluttered scenes[C]//Proceedings of the 2021 IEEE International Conference on Robotics and Automation. Piscataway: IEEE, 2021: 13438-13444.
[21] FANG H S, WANG C X, GOU M H, et al. GraspNet-1Billion: a large-scale benchmark for general object grasping[C]//Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2020: 11441-11450.
[22] WANG C X, FANG H S, GOU M H, et al. Graspness discovery in clutters for fast and accurate grasp detection[C]//Proceedings of the 2021 IEEE/CVF International Conference on Computer Vision. Piscataway: IEEE, 2022: 15944-15953.
[23] CHOY C, GWAK J, SAVARESE S. 4D spatio-temporal Conv-Nets: minkowski convolutional neural networks[C]//Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2020: 3070-3079.
[24] MA H X, HUANG D. Towards scale balanced 6-DoF grasp detection in cluttered scenes[C]//Proceedings of the Conference on Robot Learning, 2022.
[25] LIU J R, ZHANG R, FANG H S, et al. Target-referenced reactive grasping for dynamic objects[C]//Proceedings of the 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2023: 8824-8833.
[26] CHU F J, XU R N, VELA P A. Real-world multiobject, multigrasp detection[J]. IEEE Robotics and Automation Letters, 2018, 3(4): 3355-3362.
[27] GOU M H, FANG H S, ZHU Z D, et al. RGB matters: learning 7-DoF grasp poses on monocular RGBD images[C]//Proceedings of the 2021 IEEE International Conference on Robotics and Automation. Piscataway: IEEE, 2021: 13459-13466.