基于退化四元数注意力机制的轻量化Transformer去雨网络

doi:10.3778/j.issn.1002-8331.2307-0100

摘要/Abstract

摘要： 现有主流图像去雨方法专注于提升去雨性能，而忽略了网络计算开销过大的问题。少数轻量化网络的研究只局限于修改网络结构来简化网络计算。针对上述问题，利用退化四元数可以获得更多图像先验信息的特性提出了一个基于退化四元数图像去雨网络。网络使用退化四元数Swin-Transformer块（reduced biquaternion Swin-Transformer block，RQSTB）作为主要特征提取模块。其中设计了使用基于退化四元数多头注意力机制的Transformer块提取全局特征信息，同时穿插使用退化四元数多尺度卷积模块提取局部多尺度特征信息，用以弥补Transformer缺乏卷积神经网络自带的一些归纳偏置的缺陷。经实验证明，该方法在网络参数和计算复杂度方面都优于很多现有的图像去雨方法，并且在去雨性能方面也达到了先进的水平，无论是从定量还是定性的指标来看，都展现了显著的效果。

关键词: 图像去雨, 退化四元数网络, Transformer, 轻量化

Abstract: The existing mainstream image deraining methods focus on improving the deraining performance but ignore the problem of excessive network computation overhead. A few lightweight network studies are restricted to changing the network’s structure to reduce computation. The capability of the reduced biquaternion (RQ) to obtain more a priori information is used to propose a reduced biquaternion image deraining network as a solution to the above issue. The network’s primary feature extraction module is the reduced biquaternion Swin-Transformer block (RQSTB). The RQSTB incorporates the reduced biquaternion Transformer block, which utilizes the reduced biquaternion multi-headed attention mechanism to extract global feature information. Additionally, the reduced biquaternion multi-scale convolution module is interleaved to capture local multi-scale feature information. This combination compensates for the inherent lack of CNN’s inductive biases that are absent in Transformer. It is experimentally demonstrated that this method achieves advanced levels of deraining performance, outperforming most existing image rain removal methods in terms of network parameters and computational complexity and demonstrating significant results in terms of both quantitative and qualitative metrics.

Key words: image deraining, reduced biquaternion network, Transformer, lightweight

熊贡鹤, 陈飞龙, 孙成立, 郭桥生. 基于退化四元数注意力机制的轻量化Transformer去雨网络[J]. 计算机工程与应用, 2024, 60(19): 250-258.

XIONG Gonghe, CHEN Feilong, SUN Chengli, GUO Qiaosheng. Lightweight Transformer Deraining Network Based on Reduced Biquaternion Attention Mechanism[J]. Computer Engineering and Applications, 2024, 60(19): 250-258.

参考文献

[1] XU K, TIAN X, YANG X, et al. Intensity-aware single-image deraining with semantic and color regularization[J]. IEEE Transactions on Image Processing, 2021, 30: 8497-8509.
[2] FU X, XIAO J, ZHU Y, et al. Continual image deraining with hypergraph convolutional networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023, 45: 9534-9551.
[3] HAO Z, GAI S, LI P. Multi-scale self-calibrated dual-attention lightweight residual dense deraining network based on monogenic wavelets[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2023, 33(6): 2642-2655.
[4] CUI X, WANG C, REN D, et al. Semi-supervised image deraining using knowledge distillation[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2022, 32: 8327-8341.
[5] 方思严, 刘斌. 小波分频自注意力Transformer图像去雨网络[J]. 计算机工程与应用, 2024,60(6): 259-273.
FANG S Y, LIU B. Wavelet frequency division self-attention Transformer image deraining network[J]. Computer Engineering and Applications ,2024, 60(6): 259-273.
[6] LI P, GAI S. Single image deraining using multi-scales context information and attention network[J]. Journal of Visual Communication and Image Representation, 2022, 90: 103695.
[7] HUANG Z X, ZHANG J. Contrastive unfolding deraining network[J]. IEEE Transactions on Neural Networks and Learning Systems, 2024, 35（4）：5155-5169.
[8] ZAMIR S W, ARORA A, KHAN S H, et al. Multi-stage progressive image restoration[C]//Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2021: 14816-14826.
[9] WANG Z, CUN X, BAO J, et al. Uformer: a general U-shaped Transformer for image restoration[C]//Proceedings of the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2021: 17662-17672.
[10] REN D, ZUO W, HU Q, et al. Progressive image deraining networks: a better and simpler baseline[C]//Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2019: 3932-3941.
[11] LIANG Y, ANWAR S, LIU Y. DRT: a lightweight single image deraining recursive Transformer[C]//Proceedings of the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), 2022: 588-597.
[12] GAI S, HUANG X. Reduced biquaternion convolutional neural network for color image processing[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2021, 32: 1061-1075.
[13] ZHU X, XU Y, XU H, et al. Quaternion convolutional neural networks[C]//Proceedings of the European Conference on Computer Vision (ECCV), 2018: 645-661.
[14] DOSOVITSKIY A, BEYER L, KOLESNIKOV A, et al. An image is worth 16×16 words: Transformers for image recognition at scale[J]. arXiv:2010.11929, 2021.
[15] YANG W, TAN R T, FENG J, et al. Deep joint rain detection and removal from a single image[C]//Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016: 1685-1694.
[16] CHEN L, LU X, ZHANG J, et al. HINet: half instance normalization network for image restoration[C]//Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), 2021: 182-192.
[17] ZHANG H, PATEL V M. Density-aware single image de-raining using a multi-stream dense network[C]//Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2018: 695-704.
[18] YASARLA R, PATEL V M. Uncertainty guided multi-scale residual learning-using a cycle spinning CNN for single image de-raining[C]//Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2019: 8397-8406.
[19] LI X, WU J, LIN Z, et al. Recurrent squeeze-and-excitation context aggregation net for single image deraining[C]//Proceedings of the European Conference on Computer Vision, 2018.