Abstract: This paper presents a method for individually describing the video segments with open eyes and those with blink and coarse-to-fine blink detection based on two-stage cascade model composed of sequence-level and frame-level. In sequence-level detection phase, the video segment is condensed and CNN feature is extracted. Optical flow between frames is accumulated and expressed as dynamic feature by Bag of Feature（BoF）. The above two features are fused to discover whether blinks exist in the current segment by classification. In frame-level detection phase, the blink motion is described precisely to describe every frame may contain blink by extracting multi-mode features. Eye closity is calculated by random regression forest and the precise blink localization is obtained. Experiments on different datasets show the proposed method improves the robustness of blink detection under uncontrolled real environment and reaches an increased performance in correct rate, convergence and speed are under the considerable computational complexity. Compared with the other current methods, the method has significant value in real applications.

Key words: sequence-level detection, frame-level detection, multi-mode feature, random regression forest

摘要： 提出一种采用级联的序列级和帧级两层检测模型，分别处理持续睁眼和眨眼的视频片段，由粗到精渐进地检测眨眼的方法。在序列级初步检测阶段，将视频表示为浓缩图并提取卷积神经网络（CNN）特征。累计视频中的帧间光流，采用特征包模型表示为动态特征，融合以上两种特征进行分类，将持续睁眼和可能存在眨眼的视频快速分开。在帧级精确检测阶段，精细刻画眨眼过程，提取多模式特征描述可能眨眼的每帧图像，通过随机回归森林计算眼睛开合度，最终完成眨眼过程的精确定位。在两个数据库上进行了实验，将该算法与其他算法进行了定量比较，结果表明该算法在鲁棒性、正确率和处理速度等方面都达到很好的性能，具有明显的实用价值。

关键词: 序列级检测, 帧级检测, 多模式特征, 随机回归森林