Adaptive IQA Threshold for NLM Multi-Frame Super-Resolution Method

doi:10.3778/j.issn.1002-8331.2101-0523

Abstract

Abstract: Multi-frame image super-resolution（SR） algorithm uses complementary information between different low resolution images to reconstruct a high resolution image. Traditional SR method based on non-local means（NLM） is highly dependent on experience and subjective evaluation during the selection process of the iteration numbers and the best SR results. To solve these two problems, a NLM super-resolution method based on image quality assessment（IQA） adaptive threshold is proposed. The paper designs an evaluation index to assess the quality of SR result, and introduces this index to the NLM SR algorithm：on one hand, it functions as a threshold to determine the iterative convergence criteria of the algorithm; on the other hand, it helps selecting the SR result with the best reconstruction performance as assessment index. Several experiments prove that the proposed algorithm can ensure its robustness and greatly improve the computational efficiency of the NLM super-resolution reconstruction method.

Key words: super-resolution（SR）, non-local means（NLM）, image quality assessment（IQA）, adaptive threshold

摘要： 序列图像超分辨率（super resolution，SR）算法可以利用多帧低分辨率图像之间的互补信息重建出一张高分辨率结果。传统非局部均值（non-local means，NLM）超分辨率重建方法的迭代次数选取和最佳SR重建结果筛选过程高度依赖使用者经验值和主观评价，这极大地增加了算法复杂度，降低了算法的鲁棒性。为了解决这两个问题，提出一种基于图像质量评价（image quality assessment，IQA）自适应阈值的NLM超分辨重建算法。通过设计一种SR重建结果质量评价指标，将该指标引入到NLM重建算法中：一方面作为阈值，用以确定算法迭代收敛条件；另一方面作为评价标准，用以筛选多个输出结果中重建效果最佳的高分辨率图像。实验结果表明，提出的算法能在有效保证鲁棒性的同时，极大地提升NLM超分辨率重建算法的运算效率。

关键词: 超分辨率, 非局部均值, 图像质量评价, 自适应阈值

WEI Zixian, XIONG Zhengqiang, MAO Yutong, SUN Tao. Adaptive IQA Threshold for NLM Multi-Frame Super-Resolution Method[J]. Computer Engineering and Applications, 2022, 58(13): 249-256.

韦子先, 熊正强, 毛昱童, 孙涛. 自适应IQA阈值序列图像NLM超分辨重建方法[J]. 计算机工程与应用, 2022, 58(13): 249-256.

References

[1] LERTRATTANAPANICH S，BOSE N K.High resolution image formation from low resolution frames using delaunay triangulation[J].IEEE Transactions on Image Processing，2002，11（12）：1427-1441.
[2] FARSIUA S，ROBINSONA D，ELADB M，et al.Robust shift and add approach to super-resolution[J].International Society for Optical Engineering，2003，5203：121-130.
[3] RASTI P，DEMIREL H，ANBARJAFARI G.Improved iterative back projection for video super-resolution[C]//Proceedings of the 22nd Signal Processing and Communications Applications Conference（SIU），2014.
[4] GAO J，STANTON A，NAGHIZADEH M，et al.Convergence improvement and noise attenuation considerations for beyond alias projection onto convex sets reconstruction[J].Geophysical Prospecting，2013，61：138-151.
[5] FANG Y，CAO J，WANG J，et al.HTPR*-tree：An efficient index for moving objects to support predictive query and partial history query[C]//Proceedings of International Conference on Web-Age Information Management，2011.
[6] PATANAVIJIT V，JITAPUNKUL S.A Lorentzian stochastic estimation for a robust iterative multiframe super-resolution reconstruction with Lorentzian-Tikhonov regularization[J].Eurasip Journal on Advances in Signal Processing，2007（2）：1-21.
[7] ZENG W L，LU X B.A robust variational approach to super-resolution with nonlocal TV regularisation term[J]. Journal of Photographic Science，2013，61（2）：268-278.
[8] ZENG X，YANG L.A robust multiframe super-resolution algorithm based on half-quadratic estimation with modified BTV regularization[J].Digital Signal Processing，2013，23：98-109.
[9] Buades A，Coll B，Morel J M.A non-local algorithm for image denoising[C]//Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition，2005.
[10] PROTTER M，ELAD M，TAKEDA H，et al.Generalizing the nonlocal-means to super-resolution reconstruction[J].IEEE Transactions on Image Processing，2009，18（1）：36.
[11] MITTAL A，MOORTHY A K，BOVIK A C.No-reference image quality assessment in the spatial domain[J].IEEE Transactions on Image Processing，2012，21（12）：4695-4708.
[12] 张渊.基于NLM的多帧图像超分辨算法研究[D].西安：西安电子科技大学，2015.
ZHANG Y.The research on super-resolution reconstruction algorithm for multi-frame images based on NLM[D].Xi’an：Xidian University，2015.
[13] COOK S.CUDA programming：A developer’s guide to parallel computing with GPUs[M].[S.l.]：Elsevier，MK，2012.
[14] 李瑶，孙涛，黄驰，等.序列图像的非局部均值超分辨率重建算法及GPU实现[J].计算机应用研究，2016，33（7）：2201-2205.
LI Y，SUN T，HUANG C，et al.Non-local means reconstruction of image sequences and GPU implementation[J].Application Research of Computers，2016，33（7）：2201-2205.
[15] 叶双清，杨晓梅.基于小波变换和非局部平均的超分辨率图像重建[J].计算机应用，2014，34（4）：1182-1186.
YE S Q，YANG X M.Super resolution image reconstruction based on wavelet transform and non-local means[J].Journal of Computer Applications，2014，34（4）：1182-1186.