Abstract: The learning feature detector applies neural network to detect and match image feature points. Its network parameters are usually trained by optimizing the matching accuracy of the low-level vision. However, in advanced vision tasks, the improvement of low-level matching accuracy may not bring better performance. To address this problem, a double-loss error strategy reinforcement learning method is proposed. On the one hand, the feature points and descriptors obtained by the learning invariant feature transform（LIFT） are represented in probabilistic form. The relative pose between images is estimated, and the pose error is obtained by comparing it with the real pose. On the other hand, the error of descriptor between the matching feature points is estimated using the geometric property of epipolar constraint between matched images. Then, the LIFT is optimized based on the above two loss errors, and finally the neural network parameters are learned. The H-Patches dataset and self-made dataset are used in the experiment, and the images are fed into the LIFT feature detector and vision pipeline, and the neural network parameters are trained in an end-to-end manner. Experimental results show that the proposed algorithm significantly improves the matching accuracy of feature points.

Key words: reinforcement learning, epipolar constraint, feature descriptor, neural network, loss function, image matching

摘要： 学习型特征检测器利用神经网络来检测和匹配图像特征点，其网络参数通常通过优化低层视觉的匹配准确率而训练得到，然而在高级视觉任务中，低层图像配准率的提升未必能带来更佳性能。针对该问题，提出一种双损失误差策略下的强化学习方法，一方面，将学习不变特征变换（LIFT）所得到的特征点和描述符以概率形式表示，估算出图像间的相对位姿，并与真实位姿比较获得位姿误差。另一方面，利用匹配图像间极线约束的几何性质，估算出匹配特征点间描述子的误差。然后基于上述两种损失误差优化LIFT，最终学习得到神经网络参数。实验中使用H-Patches数据集和自制数据集，将图像输送到LIFT特征检测器和视觉管道中，以端到端的方式训练神经网络参数。实验结果表明，该算法显著提高了特征点的匹配精度。

关键词: 强化学习, 极线约束, 特征描述子, 神经网络, 损失函数, 图像匹配