Abstract:

A graph-based Human-Object Interactions（HOIs） recognition method is proposed. In order to effectively represent the rich interactions between humans and objects in static images, the graph structure with powerful relational modeling capabilities is used to generate corresponding human-object interaction graph for the image. Considering the good performance of context information on various image recognition tasks, for utilizing the context information in the image, a Feature Processing Network（FPNet） that introduces an attention mechanism is proposed. The detection and recognition of real HOIs is done through a Graph Attention Network（GAT）. The method is validated on the V-COCO dataset and HICO-DET dataset, and compared with other methods. The results show that the proposed method has good results.

Key words: human-object interactions, context, attention mechanism, Graph Attention Network（GAT）

摘要：

提出了一种基于图的人与物体的交互（Human-Object Interactions，HOIs）识别方法。为了对静态图像中人与物体间丰富的交互关系进行有效的表示，采用具有强大关系建模能力的图结构为图像生成对应的人-物交互关系图。为了对图像中上下文（context）信息加以利用，提出了引入注意力机制的特征处理网络（Feature Processing Network，FPNet）。通过图注意力（Graph Attention Network，GAT）网络完成对真实的HOIs的检测和识别。该方法在V-COCO数据集与HICO-DET数据集上进行了验证，并与其他方法进行了比较，结果表明该方法具有较好的效果。

关键词: 人-物交互, 上下文, 注意力机制, 图注意力网络