关键词: 计算机视觉, 视频摘要, 注意力机制, 局部奖励函数

Abstract:

Video summarization aims to shorten the length of the video while preserving the main content, eminently saving time of browsing videos. A key step of video summarization is to evaluate the performance of generated summaries, whereas most existing methods focus on evaluating it based on the whole video. However, evaluation based on the entire video sequence is computationally expensive, especially for long videos. Moreover, the evaluation of the generated summary on the entire video often ignores the inherent temporal relationship of the video data, which leads to the lack of logic of the storyline. It thereby proposes a novel framework for video summarization called Attentive Local Reward Summarization Network（ALRSN）. To be precise, the model performs frame-level important score predictions through a self-attention mechanism. To evaluate the performance of generated summaries, it further designs a local reward function that jointly accounts for both the local diversity and local representativeness. The generated summary maps to the original video and evaluates the performance in a local scope, therefore it has the temporal relationship. In addition, the local reward function encourages the model to produce a more diverse and representative summary in the local scope, thereby obtaining a higher reward. The comprehensive experiment on two benchmark datasets, SumMe and TvSum, shows that the ALRSN model is superior to the state-of-the-art methods.

Key words: computer vision, video summarization, attention mechanism, local reward function