DCN：Image Quality Assessment Network of Dual-Channel Dense Hadamard Product

doi:10.3778/j.issn.1002-8331.2103-0560

Abstract

Abstract: Image quality assessment（IQA） is becoming more and more important as the human needs for high quality images become more and more urgent. The non-reference authentic distortion assessment is faced with great challenges due to the complexity of distortion and content diversity. In order to obtain more accurate and effective quality characteristics, the dual-channel dense Hadamard convolution image quality evaluation network（DCN） is proposed. In DCN, the deep convolution model of Infection-ResNet-V2 is used as the backbone network to extract features, and the designed dual-channel converged network is used as the score evaluation network, which is finally mapped to the objective quality score. The score evaluation network is composed of a convolution feature extraction branch and a multilayer perceptron branch in parallel. In the multilayer perceptron branch, a dense hadamard product modul（DHPM） is proposed. Through the Hadamard product, low-level features and high-level features are integrated to play the role of feature adaptation and high-level expression. Experimental results on the open data set Koniq-10K show that the Spearman rank order correlation coefficient（SROCC） and Pearson linear correlation coefficient（PLCC） of the network test are 0.922 and 0.938 respectively.

Key words: image quality assessment, deep learning, computer vision, dual-channel structure, Hadamard product

摘要： 随着人类对高质量图像的需求日益紧迫，客观画质评价（image quality assessment，IQA）的研究日趋重要，其中的无参考真实失真评估，面临失真的复杂性和内容多样性的巨大挑战。为了获取更加准确有效的质量特征，提出了一种双通道密集哈达玛卷积的画质评价网络（dual-channel network，DCN），其以深度卷积模型Inception-ResNet-v2为骨干网络提取特征，将设计的双通道融合网络为分数评估网络，最后映射到客观质量分数。分数评估网络由卷积特征提取分支和多层感知机分支并联组成，将提出的密集哈达玛卷积模块（dense Hadamard product module，DHPM）应用到多层感知机分支中，通过哈达玛乘积将低层特征与高层特征融合，发挥特征自适应和高级表达的作用。在公开数据集KonIQ-10k上的实验结果表明，该网络测试的斯皮尔曼秩相关系数（spearman rankorder correlation coefficient，SROCC）为0.922，皮尔森线性相关系数（Pearson linear correlation coefficient，PLCC）达到0.938。

关键词: 图像质量评价, 深度学习, 计算机视觉, 双通道结构, 哈达玛卷积

YANG Xiaodong, HAN Zhenqi, LIU Lizhuang, ZHAO Dan. DCN：Image Quality Assessment Network of Dual-Channel Dense Hadamard Product[J]. Computer Engineering and Applications, 2022, 58(21): 243-249.

杨晓东, 韩振奇, 刘立庄, 赵丹. DCN：双通道密集哈达玛卷积的画质评价网络[J]. 计算机工程与应用, 2022, 58(21): 243-249.

References

[1] KIM J，ZENG H，GHADIYARAM D，et al.Deep convolutional neural models for picture-quality prediction：challenges and solutions to data-driven image quality assessment[J].IEEE Signal Processing Magazine，2017，34（6）：130-141.
[2] DODGE S，KARAM L.Understanding how image quality affects deep neural networks[C]//2016 Eighth International Conference on Quality of Multimedia Experience（QoMEX），2016：1-6.
[3] LIU A，LIN W，NARWARIA M.Image quality assessment based on gradient similarity[J].IEEE Transactions on Image Processing，2011，21（4）：1500-1512.
[4] WANG Z，BOVIK A C，SHEIKH H R，et al.Image quality assessment：from error visibility to structural similarity[J].IEEE Transactions on Image Processing，2004，13（4）：600-612.
[5] ZHANG Lin，ZHANG Lei，MOU Xuanqin，et al.FSIM：a feature similarity index for image quality assessment[J].IEEE Transactions on Image Processing，2011，20（8）：2378-2386.
[6] XUE W，ZHANG L，MOU X，et al.Gradient magnitude similarity deviation：a highly efficient perceptual image quality index[J].IEEE Transactions on Image Processing，2013，23（2）：684-695.
[7] DING L，HUANG H，ZANG Y.Image quality assessment using directional anisotropy structure measurement[J].IEEE Transactions on Image Processing，2017，26（4）：1799-1809.
[8] LIANG Y，WANG J，WAN X，et al.Image quality assessment using similar scene as reference[C]//European Conference on Computer Vision.Cham：Springer，2016：3-18.
[9] GAO F，WANG Y，LI P，et al.Deepsim：deep similarity for image quality assessment[J].Neurocomputing，2017，257：104-114.
[10] KIM J，LEE S.Deep learning of human visual sensitivity in image quality assessment framework[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2017：1676-1684.
[11] WANG Z，SIMONCELLI E P.Reduced-reference image quality assessment using a wavelet-domain natural image statistic model[C]//Proceedings of 5666，Human Vision and Electronic Imaging X，2005：149-159.
[12] SOUNDARARAJAN R，BOVIK A C.RRED indices：reduced reference entropic differencing for image quality assessment[J].IEEE Transactions on Image Processing，2011，21（2）：517-526.
[13] MOORTHY A K，BOVIK A C.A two-step framework for constructing blind image quality indices[J].IEEE Signal Processing Letters，2010，17（5）：513-516.
[14] MITTAL A，SOUNDARARAJAN R，BOVIK A C.Making a “Completely Blind” image quality analyzer[J].IEEE Signal Processing Letters，2012，20（3）：209-212.
[15] MITTAL A，MOORTHY A K，BOVIK A C.Blind/referenceless image spatial quality evaluator[C]//2011 Conference Record of the Forty-Fifth Asilomar Conference on Signals，Systems and Computers（ASILOMAR），2011：723-727.
[16] LIU L，LIU B，HUANG H，et al.No-reference image quality assessment based on spatial and spectral entropies[J].Signal Processing：Image Communication，2014，29（8）：856-863.
[17] XU J，YE P，LI Q，et al.Blind image quality assessment based on high order statistics aggregation[J].IEEE Transactions on Image Processing，2016，25（9）：4444-4457.
[18] BIANCO S，CELONA L，NAPOLETANO P，et al.On the use of deep learning for blind image quality assessment[J].Signal，Image and Video Processing，2018，12（2）：355-362.
[19] LIU X，VAN DE WEIJER J，BAGDANOV A D.Rankiqa：learning from rankings for no-reference image quality assessment[C]//Proceedings of the IEEE International Conference on Computer Vision，2017：1040-1049.
[20] ZENG H，ZHANG L，BOVIK A C.A probabilistic quality representation approach to deep blind image quality prediction[J].arXiv：1708.08190，2017.
[21] PAN D，SHI P，HOU M，et al.Blind predicting similar quality map for image quality assessment[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2018：6373-6382.
[22] LI D，JIANG T，LIN W，et al.Which has better visual quality：the clear blue sky or a blurry animal?[J].IEEE Transactions on Multimedia，2018，21（5）：1221-1234.
[23] ZHANG W，MA K，YAN J，et al.Blind image quality assessment using a deep bilinear convolutional neural network[J].IEEE Transactions on Circuits and Systems for Video Technology，2020，30（1）：36-47.
[24] ZHU H，LI L，WU J，et al.MetaIQA：deep meta-learning for no-reference image quality assessment[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition，2020：14143-14152.
[25] SU S，YAN Q，ZHU Y，et al.Blindly assess image quality in the wild guided by a self-adaptive hyper network[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition，2020：3667-3676.
[26] HOSU V，LIN H，SZIRANYI T，et al.KonIQ-10k：an ecologically valid database for deep learning of blind image quality assessment[J].IEEE Transactions on Image Processing，2020，29：4041-4056.
[27] LI D，JIANG T，JIANG M.Norm-in-norm loss with faster convergence and better performance for image quality assessment[C]//Proceedings of the 28th ACM International Conference on Multimedia，2020：789-797.
[28] SZEGEDY C，IOFFE S，VANHOUCKE V，et al.Inception-v4，inception-resnet and the impact of residual connections on learning[C]//Proceedings of the AAAI Conference on Artificial Intelligence，2017.
[29] LIU G，WANG J.Dendrite net：a white-box module for classification，regression，and system identification[J].arXiv：2004.03955，2020.
[30] HE K，ZHANG X，REN S，et al.Deep residual learning for image recognition[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2016：770-778.
[31] HUANG G，LIU Z，VAN DER MAATEN L，et al.Densely connected convolutional networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2017：4700-4708.
[32] ZHAO H，JIA J，KOLTUN V.Exploring self-attention for image recognition[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition，2020：10076-10085.