神经辐射场恢复水下图像失真

doi:10.3778/j.issn.1002-8331.2401-0357

摘要/Abstract

摘要： 神经辐射场（neural radiance fields，NeRF）是一个备受关注的新兴研究领域，旨在解决新视角合成问题。然而，将其应用于水下环境时存在诸多挑战，如淹没在波形水面下的静态场景图像时常呈现严重的非刚性失真，这主要是由于水面波动和折射导致的视觉扭曲。为了解决水下物体图像失真问题，深入分析波形水面的光学特性以及水下物体表面的失真情况，提出了一种针对水下场景的NeRF框架。自制了一种数据采集装置，将相机固定在水面上方，根据水流在时间和空间具有周期性的物理特性，仿真出多视角的虚拟相机，有效解决了水下数据采集难题。利用图像光流恢复水面波形。基于斯涅尔定律校正采样策略，通过重定义光线追踪过程中的起点和方向，渲染出校正后的水下图像。利用真实水下场景数据集对该方法进行了定性和定量的评估，结果表明该方法能够有效消除水下图像失真，且该方法在水下图像三维重建方面的表现优于现有的算法，为水下新视角图像合成提供了新的思路和方法。

关键词: 神经辐射场, 水下物体, 光流法, 斯涅尔定律

Abstract: Neural radiance fields (NeRF) represents a burgeoning area of research that has garnered significant attention, with a primary focus on addressing challenges related to new perspective synthesis. However, the application of NeRF to underwater environments poses several challenges. In underwater scenarios, static scene images submerged under undulating water often exhibit pronounced non-rigid distortions. These distortions primarily arise from visual distortions caused by fluctuations and refraction at the water surface. To tackle the issue of underwater object image distortion, this paper conducts an in-depth analysis of the optical characteristics in underwater flow environments and the distortion of underwater object surfaces. The paper proposes a NeRF framework tailored for underwater scenes. Initially, a self-made data collection device positions the camera above the water surface, addressing the challenge of underwater data collection by simulating a multi-view virtual camera based on the periodic physical characteristics of water flow in both time and space. Subsequently, the optical flow of the images is utilized to restore the waveform shape of the water surface. Finally, employing Snell’s Law correction sampling strategy, the corrected underwater image is rendered by redefining the starting point and direction in the ray tracing process. Qualitative and quantitative evaluations of the method are conducted using a real underwater scene dataset. The results demonstrate that this approach significantly mitigates underwater image distortion, outperforming existing methods in the three-dimensional reconstruction of underwater images. The algorithm introduces novel ideas and methods for synthesizing new perspectives in underwater imagery.

Key words: neural radiance fields (NeRF), underwater objects, optical flow estimation, Snell’s law

华喜锋, 黄志勇, 杨晨龙, 姚玉. 神经辐射场恢复水下图像失真[J]. 计算机工程与应用, 2025, 61(9): 296-303.

HUA Xifeng, HUANG Zhiyong, YANG Chenlong, YAO Yu. Neural Radiance Fields for Restoring Distorted Underwater Images[J]. Computer Engineering and Applications, 2025, 61(9): 296-303.

参考文献

[1] JAIN S. DeepSeaNet: improving underwater object detection using EfficientDet[J]. arXiv:2306.06075, 2023.
[2] WANG S, WU W, WANG X, et al. Underwater optical image object detection based on YOLOv7 algorithm[C]//Proceedings of the OCEANS 2023-Limerick, 2023: 1-5.
[3] WANG G, HWANG J N, WILLIAMS K, et al. Closed-loop tracking-by-detection for ROV-based multiple fish tracking[C]//Proceedings of the 2016 ICPR 2nd Workshop on Computer Vision for Analysis of Underwater Imagery (CVAUI), 2016: 7-12.
[4] LIN H Y, TSENG S L, LI J Y. SUR-Net: a deep network for fish detection and segmentation with limited training data[J]. IEEE Sensors Journal, 2022, 22(18): 18035-18044.
[5] LI Z, MUREZ Z, KRIEGMAN D, et al. Learning to see through turbulent water[C]// Proceedings of the 2018 IEEE Winter Conference on Applications of Computer Vision (WACV), 2018: 512-520.
[6] QIAN Y, ZHENG Y, GONG M, et al. Simultaneous 3D reconstruction for water surface and underwater scene[C]//Proceedings of the European Conference on Computer Vision (ECCV), 2018: 754-770.
[7] XIONG J, HEIDRICH W. In-the-wild single camera 3D reconstruction through moving water surfaces[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision, 2021: 12558-12567.
[8] MILDENHALL B, SRINIVASAN P, TANCIK M, et al. NeRF: representing scenes as neural radiance fields for view synthesis[J]. Communications of the ACM, 2021, 65(1): 99-106.
[9] CURLESS B, LEVOY M. A volumetric method for building complex models from range images[C]//Proceedings of the 23rd Annual Conference on Computer Graphics and Interactive Techniques, 1996: 303-312.
[10] MüLLER T, EVANS A, SCHIED C, et al. Instant neural graphics primitives with a multiresolution hash encoding[J]. ACM Transactions on Graphics (ToG), 2022, 41(4): 1-15.
[11] CHEN A, XU Z, GEIGER A, et al. TensoRF: tensorial radiance fields[C]//Proceedings of the European Conference on Computer Vision. Cham: Springer Nature Switzerland, 2022: 333-350.
[12] BARRON J T, MILDENHALL B, VERBIN D, et al. Mip-NeRF 360: unbounded anti-aliased neural radiance fields[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022: 5470-5479.
[13] GARBIN S J, KOWALSKI M, JOHNSON M, et al. FastNeRF: high-fidelity neural rendering at 200FPS[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision, 2021: 14346-14355.
[14] 马汉声, 祝玉华, 李智慧, 等. 神经辐射场多视图合成技术综述[J].计算机工程与应用, 2024, 60(4): 21-38.
MA H S, ZHU Y H, LI Z H, et al. Survey of neural radiance fields for multi-view synthesis technologies[J]. Computer Engineering and Applications ,2024, 60(4): 21-38.
[15] KERBL B, KOPANAS G, LEIMKüHLER T, et al. 3D Gaussian splatting for real-time radiance field rendering[J]. ACM Transactions on Graphics, 2023, 42(4):1-14.
[16] SETHURAMAN A V, RAMANAGOPAL M S, SKINNER K A.WaterNeRF:neural radiance fields for underwater scenes[C]//Proceedings of the OCEANS 2023-MTS/IEEE US Gulf Coast, 2023: 1-7.
[17] LEVY D, PELEG A, PEARL N, et al. SeaThru-NeRF: neural radiance fields in scattering media[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023: 56-65.
[18] AKKAYNAK D, TREIBITZ T. A revised underwater image formation model[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018: 6723-6732.
[19] AKKAYNAK D, TREIBITZ T, SHLESINGER T, et al. What is the space of attenuation coefficients in underwater computer vision?[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017: 4931-4940.
[20] SCHONBERGER J L, FRAHM J M. Structure-from-motion revisited[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016: 4104-4113.
[21] KOZLOV I, RZHANOV Y. Uncertainty in 3D reconstruction of underwater objects due to refraction[C]//Proceedings of the OCEANS 2017-Anchorage, 2017: 1-4.
[22] TIAN Y, NARASIMHAN S G. Seeing through water: image restoration using model-based tracking[C]//Proceedings of the 2009 IEEE 12th International Conference on Computer Vision, 2009: 2303-2310.
[23] OREIFEJ O, SHU G, PACE T, et al. A two-stage reconstruction approach for seeing through water[C]//Proceedings of the Conference on Computer Vision and Pattern Recognition (CVPR), 2011: 1153-1160.
[24] SUN D, YANG X, LIU M Y, et al. PWC-Net: CNNs for optical flow using pyramid, warping, and cost volume[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018: 8934-8943.
[25] TEED Z, DENG J. RAFT: recurrent all-pairs field transforms for optical flow[C]//Proceedings of the 16th European Conference on Computer Vision(ECCV 2020), Glasgow, UK, August 23-28, 2020: 402-419.
[26] ZHAO S, SHENG Y, DONG Y, et al. MaskFlownet: asymmetric feature matching with learnable occlusion mask[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020: 6278-6287.
[27] DOSOVITSKIY A, FISCHER P, ILG E, et al. FlowNet: learning optical flow with convolutional networks[C]//Proceedings of the IEEE International Conference on Computer Vision, 2015: 2758-2766.