Computer Engineering and Applications ›› 2022, Vol. 58 ›› Issue (24): 284-290.DOI: 10.3778/j.issn.1002-8331.2106-0293

• Engineering and Applications • Previous Articles     Next Articles

Exploring Interpretability of Attention Mechanism Based on Mobility Pattern

WENG Xiaoxiong, TIAN Dan, QIN Zhenlin1, LUO Ruifa   

  1. 1.School of Civil Engineering and Transportation, South China University of Technology, Guangzhou 510630,  China
    2.Shenzhen Genvict Technologies Co. Ltd., Shenzhen, Guangdong 518000, China
  • Online:2022-12-15 Published:2022-12-15

基于出行模式的注意力机制可解释性探索

翁小雄,田丹,覃镇林,罗瑞发   

  1. 1.华南理工大学 土木与交通学院,广州 510630
    2.深圳市金溢科技股份有限公司,广东 深圳 518000

Abstract: In order to explore the interpretability of the deep attentive model in metro travel prediction tasks, an attention weight erasure method and an interpretability evaluation framework based on travel patterns are proposed. a predictive model is built by the proposed metro travel deep attention framework, Guangzhou Metro Yangchengtong data is used to construct three travel sequence datasets of different lengths for model training and verification, and achieves more than 70% accuracy. By the experimental results of erasing the attention weight of a single pattern, It’s found that the mobility pattern that erases the maximum attention weight can significantly affect the model prediction results than the random pattern. However, most samples do not change the predicted results. That is, the interpretability information provided by the attention mechanism is limited under this condition. And the amount of information decreases as the length of the sequence increases. Finally, the experimental results of erasing the attention weights of a set of patterns show that the mobility pattern with a larger attention weight can significantly affect the model prediction results. Moreover, the model can stably assign attention weights to the important travel patterns. It means that the attention mechanism provides interpretable information in a good way under this condition. And the amount of information increases as the length of the sequence increases.

Key words: metro travel prediction, mobility pattern, attention mechanism, attention weight erasure, interpretability

摘要: 为了探索深度注意力模型在地铁出行预测任务中的可解释性,提出基于出行模式的注意力权重擦除方法和可解释性评估框架。利用提出的地铁出行深度注意力框架搭建预测模型,使用广州地铁羊城通数据构造三种不同长度出行序列数据集进行模型训练和验证,达到70%以上准确率;通过单一出行模式的注意力权重擦除实验发现,擦除最大注意力权重的出行模式比随机模式更能显著地影响模型预测结果,但大多数样本不发生预测结果的变化。即注意力机制在该条件下提供的可解释性信息是有限的,且该信息量随着序列长度增加而减小;通过一组出行模式注意力权重擦除实验结果表明,按注意力权重降序擦除能最快使模型预测结果发生变化,并且模型能稳定地对重要的出行模式的出行记录分配注意力权重,即注意力机制在该条件下较好地提供了可解释性信息,且该信息量随着序列长度增加而增大。

关键词: 地铁出行预测, 出行模式, 注意力机制, 注意力权重擦除, 可解释性