Fish Recognition Based on Hierarchical Compact Bilinear Attention Network

doi:10.3778/j.issn.1002-8331.2009-0349

Abstract

Abstract: Due to the difficulty in collecting underwater fish images, the existing datasets are mainly extracted from videos. The collected fish images have problems such as complex background environment and low pixels, making the task of fine-grained fish images recognition more difficult. To solve the above problems, a network based on spatial domain attention mechanism and hierarchical compact bilinear feature fusion is proposed. The recognition network can be trained end-to-end, and consists of two parts: the first part is a background filtering network with a spatial transformation network （STN） as the attention mechanism; the second part is a vgg16 network as a feature extractor, for the difference in the response of the fine-grained features of fish images, which is based on the high-level convolution part, three groups of features are selected for network simplification and fusion of dimensionality reduction approximation, and finally the three groups of fused features are cascaded and sent to the soft-max classifier. The feature extraction network is initialized with the parameters trained on the ImageNet dataset and further fine-tuned using the fish dataset. Through comparison and verification on the F4K fish dataset, the results show that the proposed hierarchical compact bilinear attention network（STN-H-CBP） can reduce the feature dimension and reduce the amount of calculation at the same time. The performance is comparable to existing best practices.

Key words: underwater fish recognition, spatial transformation network, hierarchical compact bilinear network

摘要： 由于水下鱼类图像采集困难，现有的数据集主要以视频提取为主，采集到的鱼类图像存在背景环境复杂、像素低下等问题，使得细粒度鱼类图像识别任务难度较大。针对上述问题，提出了一种基于空间域注意力机制和分层精简双线性特征融合的网络。识别网络可进行端到端的训练，由两部分组成：第一部分是以空间变换网络（STN）为注意力机制的背景过滤网络；第二部分以vgg16网络作为特征提取器，根据高层卷积部分对鱼类图像细粒度特征响应的差异性，选取三组特征进行降维近似的网络精简融合，最终级联三组融合的特征送入softmax分类器。特征提取网络以ImageNet数据集上训练的参数进行初始化，采用鱼类数据集进一步微调。通过在F4K鱼类数据集上的对比验证，结果表明，所提出的分层精简双线性注意力网络（STN-H-CBP）在降低特征维度减少计算量的同时，在该数据集上的表现与现有的最优方法相当。

关键词: 水下鱼类识别, 空间变换网络, 分层精简双线性网络

DONG Shaojiang, LIU Wei, CAI Weiwei, RAO Zhirong. Fish Recognition Based on Hierarchical Compact Bilinear Attention Network[J]. Computer Engineering and Applications, 2022, 58(5): 186-192.

董绍江, 刘伟, 蔡巍巍, 饶志荣. 基于分层精简双线性注意力网络的鱼类识别[J]. 计算机工程与应用, 2022, 58(5): 186-192.

References

[1] RAMOS-MUNOZ J，JESUS C D J，BERNAL-CASSOLA D，et al.Early use of marine resources by Middle/Upper Pleistocene human societies：the case of Benzú rock-shelter（northern Africa）[J].Quaternary International，2016，407：6-15.
[2] MENG L，HIRAYAMA T，OYANGI S.Underwater-drone with panoramic camera for automatic fish recognition based on deep learning[J].IEEE Access，2018，6：17880-17886.
[3] BADAWI U A，ALSMADI M K.A general fish classification methodology using meta-heuristic algorithm with back propagation classifier[J].Journal of Theoretical & Applied Information Technology，2014，66（3）：803-812.
[4] RASMUS L，HILDUR O，BJARNE K E.Shape and texture based classification of fish species[C]//Scandinavian Conference on Image Analysis，2009：745-749.
[5] WANG G，HUANG J N，WILLIAMS K，et al.Shrinking encoding with two-level codebook learning for fine-grained fish recognition[C]//Computer Vision for Analysis of Underwater Imagery，2017：31-36.
[6] SAITOH T，SHIBATA T.Feature points based fish image recognition[J].Int J Comput Inf Syst Ind Manage Appl，2016（8）：12-22.
[7] KHOTIMAH W N，ARIFIN A Z，YUNIARTI A，et al.Tuna fish classification using decision tree algorithm and image processing method[C]//International Conference on Computer，Control，Informatics and Its Applications（IC3INA），2015：126-131.
[8] TAMOU A B，BENZINOU A，NASREDDINE K，et al.Underwater live fish recognition by deep learning[C]// International Conference on Image and Signal Processing.Cham：Springer，2018：275-283.
[9] QIU C，ZHANG S，WANG C，et al.Improving transfer learning and squeeze-and-excitation networks for small-scale fine-grained fish image classification[J].IEEE Access，2018，6：78503-78512.
[10] 罗建豪，吴建鑫.基于深度卷积特征的细粒度图像分类研究综述[J].自动化学报，2017，43（8）：1306-1318.
LUO J H，WU J X.Research review of fine-grained image classification based on deep convolution features[J].Acta Automatica Sinica，2017，43（8）：1306-1318.
[11] 蓝洁，周欣，何小海，等.基于跨层精简双线性网络的细粒度鸟类识别[J].科学技术与工程，2019，19（36）：240-246.
LAN J，ZHOU X，HE X H，et al.Fine-grained bird recognition based on cross-layer simplified bilinear network[J].Science Technology and Engineering，2019，19（36）：240-246.
[12] LIN D，SHEN X，LU C，et al.Deep LAC：deep localization，alignment and classification for fine-grained recog-nition[C]//IEEE Conference on Computer Vision and Pattern Recognition，2015：1666-1674.
[13] KRAUSE J，JIN H，YANG J，et al.Fine-grained recognition without part annotations[C]//IEEE Conference on Computer Vision and Pattern Recognition（CVPR），2015：5546-5555.
[14] JADERBERG M，SIMONYYAN K，ZISSERMAN A，et al.Spatial transformer networks[C]//Adv Neural Inf Process Syst，2015：2017-2025.
[15] LIN T，ROYCHOWDHURY A，MAJI S.Bilinear CNN models for fine-grained visual recognition[C]//International Conference on Computer Vision，Santiago，Chile，2015：1449-1457.
[16] GAO Y，BEIJBOM O，ZHANG N，et al.Compact bilinear pooling[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition，2016：317-326.
[17] YU C，ZHAO X，ZHENG Q，et al.Hierarchical bilinear pooling for fine-grained visual recognition[C]//Computer Vision-ECCV 2018，2018：595-610.
[18] SELVARAJU R R，COGSWELL M，DAS A，et al.Grad-Cam：visual explanations from deep networks via gradient based localization[C]//Proceedings of the IEEE International Conference on Computer Vision，2017：618-626.