基于多特征交互融合的行人过街意图预测

doi:10.3778/j.issn.1002-8331.2407-0554

摘要/Abstract

摘要： 行人意图预测对于开发安全的自动驾驶辅助系统至关重要。传统方法主要采用图卷积网络和递归架构来处理人体姿态数据，这些方法在特征融合方面存在局限性，并且依赖于行人姿态信息提取的完整性，导致行人被局部遮挡时准确性降低。为了解决以上问题，提出了多特征交互融合的行人过街意图预测模型（PEPR-Net），使用头部姿态并引入了骨架热力图信息，提高了行人被遮挡时预测的准确性，并且弥补了非欧几里得骨骼点信息与其他特征之间的互补差距。在此基础上，提出了一个多特征交互混合融合模块，用级联交叉注意力融合方法处理像素信息，级联混合融合结构处理非像素信息，形成更全面的特征表示。引入一种新的非对称双向门控循环模块（UBA-GRU）进行特征融合，采用最优融合策略实现F1分数和准确率（ACC）的最佳预测性能。在PIE数据集上进行了大量的消融实验，性能分析表明，PEPR-Net的准确率达到91%。该研究结果有望为自动驾驶系统提供更准确的行人意图预测。

关键词: 自动驾驶辅助系统, 多特征交互融合, 意图预测, 注意力机制, 骨架热力图

Abstract: Pedestrian intention prediction is very important for the development of safe advanced driver assistance systems. Traditional methods mainly use graph convolutional network and recursive architecture to process human pose data. These methods have limitations in feature fusion and rely on the integrity of pedestrian pose information extraction, resulting in lower accuracy when pedestrians are obscured. In order to solve the above problems, a multi-feature interactive fusion pedestrian crossing intention prediction module (PEPR-Net) is proposed, which uses head pose and introduces skeleton heat map information to improve the accuracy of prediction when pedestrians are blocked, and bridge the complementary gap between non-Euclidean bone point information and other features. On this basis, a multi-feature interactive hybrid fusion module is proposed, which uses cascade cross attention fusion method to process pixel information and cascade hybrid fusion structure to process non-pixel information. Finally, a new asymmetric bidirectional gated cycle module (UBA-GRU) is introduced for feature fusion, and the optimal fusion strategy is used to achieve the best predictive performance of F1 score and accuracy (ACC). A large number of ablation experiments are performed on the PIE dataset, and performance analysis shows that PEPR-Net achieves 91% accuracy. The results are expected to provide more accurate pedestrian intent predictions for autonomous driving systems.

Key words: autonomous driving assistance system, multi-feature interactive fusion, pedestrian intention, attention mechanism, skeleton heat map

杨智勇, 郭洁铷, 郭子杭, 许沁欣. 基于多特征交互融合的行人过街意图预测[J]. 计算机工程与应用, 2025, 61(21): 214-224.

YANG Zhiyong, GUO Jieru, GUO Zihang, XU Qinxin. Pedestrian Crossing Intention Prediction Based on Multi-Feature Interaction Fusion[J]. Computer Engineering and Applications, 2025, 61(21): 214-224.

参考文献

[1] 苑玉彬, 吴一全, 赵朗月, 等. 基于深度学习的无人机航拍视频多目标检测与跟踪研究进展[J]. 航空学报, 2023, 44(18): 028334.
YUAN Y B, WU Y Q, ZHAO L Y, et al. Research progress of UAV aerial video multi-object detection and tracking based on deep learning[J]. Acta Aeronautica et Astronautica Sinica, 2023, 44(18): 028334.
[2] 胡倩, 皮建勇, 胡伟超, 等. 基于改进YOLOv5的密集行人检测算法[J]. 计算机工程, 2025, 51(3): 216-228.
HU Q, PI J Y, HU W C, et al. Dense pedestrian detection algorithm based on improved YOLOv5[J]. Computer Engineering, 2025, 51(3): 216-228.
[3] RASOULI A, ROHANI M, LUO J. Bifold and semantic reasoning for pedestrian behavior prediction[C]//Proceedings of the 2021 IEEE/CVF International Conference on Computer Vision. Piscataway: IEEE, 2021: 15580-15590.
[4] HAM J S, KIM D H, JUNG N, et al. CIPF: crossing intention prediction network based on feature fusion modules for improving pedestrian safety[C]//Proceedings of the 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. Piscataway: IEEE, 2023: 3666-3675.
[5] GESNOUIN J, PECHBERTI S, STANCIULCSCU B, et al. TrouSPI-Net: spatio-temporal attention on parallel atrous convolutions and U-GRUs for skeletal pedestrian crossing prediction[C]//Proceedings of the 2021 16th IEEE International Conference on Automatic Face and Gesture Recognition. Piscataway: IEEE, 2021: 1-7.
[6] MUDULI K, SAHU V, GHOSH I. Predicting pedestrian movement in unsignalized crossings: a contextual cue-based approach[C]//Proceedings of the 2024 16th International Conference on Communication Systems & Networks. Piscataway: IEEE, 2024: 204-209.
[7] 罗朝阳, 张荣芬, 刘宇红, 等. 自动驾驶场景下的行人意图语义VSLAM[J]. 计算机工程与应用, 2024, 60(17): 107-116.
LUO Z Y, ZHANG R F, LIU Y H, et al. Pedestrian intent semantic VSLAM in automatic driving scenarios[J]. Computer Engineering and Applications, 2024, 60(17): 107-116.
[8] 桑海峰, 刘玉龙, 刘泉恺. 基于混合注意力机制的多信息行人过街意图预测[J]. 控制与决策, 2024, 39(12): 3946-3954.
SANG H F, LIU Y L, LIU Q K. Multi information pedestrian crossing intention prediction based on mixed attention mechanism[J]. Control and Decision, 2024, 39(12): 3946-3954.
[9] LING Y C, MA Z L, ZHANG Q, et al. PedAST-GCN: fast pedestrian crossing intention prediction using spatial-temporal attention graph convolution networks[J]. IEEE Transactions on Intelligent Transportation Systems, 2024, 25(10): 13277-13290.
[10] YANG B, ZHU J R, HU C, et al. Faster pedestrian crossing intention prediction based on efficient fusion of diverse intention influencing factors[J]. IEEE Transactions on Transportation Electrification, 2024, 10(4): 9071-9087.
[11] YANG B, WEI Z W, HU H Y, et al. DPCIAN: a novel dual-channel pedestrian crossing intention anticipation network[J]. IEEE Transactions on Intelligent Transportation Systems, 2024, 25(6): 6023-6034.
[12] CADENA P R G, QIAN Y Q, WANG C X, et al. Pedestrian graph: a fast pedestrian crossing prediction model based on graph convolutional networks[J]. IEEE Transactions on Intelligent Transportation Systems, 2022, 23(11): 21050-21061.
[13] ZHU D Y, ZHANG Z W, CUI P, et al. Robust graph convolutional networks against adversarial attacks[C]//Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. New York: ACM, 2019: 1399-1407.
[14] PERDANA M I, ANGGRAENI W, SIDHARTA H A, et al. Early warning pedestrian crossing intention from its head gesture using head pose estimation[C]//Proceedings of the 2021 International Seminar on Intelligent Technology and Its Applications. Piscataway: IEEE, 2021: 402-407.
[15] ZHOU Y J, GREGSON J. WHENet: real-time fine-grained estimation for wide range head pose[J]. arXiv:2005.10353, 2020.
[16] YURTKULU S C, ?AHIN Y H, UNAL G. Semantic segmentation with extended DeepLabv3 architecture[C]//Proceedings of the 2019 27th Signal Processing and Communications Applications Conference. Piscataway: IEEE, 2019: 1-4.
[17] CORDTS M, OMRAN M, RAMOS S, et al. The cityscapes dataset for semantic urban scene understanding[C]//Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2016: 3213-3223.
[18] SIMONYAN K, ZISSERMAN A. Very deep convolutional networks for large-scale image recognition[J]. arXiv:1409. 1556, 2014.
[19] SHI X J, CHEN Z R, WANG H, et al. Convolutional LSTM network: a machine learning approach for precipitation nowcasting[C]//Proceedings of the Neural Information Processing Systems, 2015
[20] DENG J, DONG W, SOCHER R, et al. ImageNet: a large-scale hierarchical image database[C]//Proceedings of the 2009 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2009: 248-255.
[21] SAK H, SENIOR A, BEAUFAYS F. Long short-term memory recurrent neural network architectures for large scale acoustic modeling[C]//Proceedings of the Interspeech 2014, 2014: 338-342.
[22] BRAKEL P, STROOBANDT D, SCHRAUWEN B. Bidirectional truncated recurrent neural networks for efficient speech denoising[C]//Proceedings of the Interspeech 2013, 2013: 2973-2977.
[23] KOTSERUBA I, RASOULI A, TSOTSOS J K. Do they want to cross? understanding pedestrian intention for behavior prediction[C]//Proceedings of the 2020 IEEE Intelligent Vehicles Symposium. Piscataway: IEEE, 2020: 1688-1693.
[24] RASOULI A, KOTSERUBA I, KUNIC T, et al. PIE: a large-scale dataset and models for pedestrian intention estimation and trajectory prediction[C]//Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision. Piscataway: IEEE, 2019: 6261-6270.
[25] KOTSERUBA I, RASOULI A, TSOTSOS J K. Benchmark for evaluating pedestrian action prediction[C]//Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, 2021: 1258-1268.
[26] NG J Y, HAUSKNECHT M, VIJAYANARASIMHAN S, et al. Beyond short snippets: deep networks for video classification[J]. arXiv:1503.08909, 2015.
[27] TRAN D, BOURDEV L, FERGUS R, et al. Learning spatiotemporal features with 3D convolutional networks[C]//Proceedings of the 2015 IEEE International Conference on Computer Vision. Piscataway: IEEE, 2016: 4489-4497.
[28] KINGMA D P, BA J. Adam: a method for stochastic optimization[J]. arXiv:1412.6980, 2014.
[29] RASOULI A, KOTSERUBA I, TSOTSOS J K. Pedestrian action anticipation using contextual feature fusion in stacked RNNs[J]. arXiv:2005.06582, 2020.
[30] DU Y, WANG W, WANG L. Hierarchical recurrent neural network for skeleton based action recognition[C]//Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2015: 1110-1118.
[31] SIMONYAN K, ZISSERMAN A. Two-stream convolutional networks for action recognition in videos[J]. arXiv:1406. 2199, 2014.
[32] BUCHMAN D, DROZDOV M, KRILAVI?IUS T, et al. Pedestrian and animal recognition using Doppler radar signature and deep learning[J]. Sensors, 2022, 22(9): 3456.
[33] WOO S, PARK J, LEE J Y, et al. CBAM: convolutional block attention module[C]//Proceedings of the European Conference on Computer Vision. Cham: Springer, 2018: 3-19.
[34] 曹昊天, 施惠杰, 宋晓琳, 等. 基于多特征融合的行人意图以及行人轨迹预测方法研究[J]. 中国公路学报, 2022, 35(10): 308-318.
CAO H T, SHI H J, SONG X L, et al. Prediction of pedestrian intention and trajectory based on multi-feature fusion[J]. China Journal of Highway and Transport, 2022, 35(10): 308-318.