Lane Detection Under Foggy Conditions via FoggyCULane Dataset

doi:10.3778/j.issn.1002-8331.2012-0138

Abstract

Abstract: In order to improve the lane detection accuracy of the deep learning algorithm under foggy condition, one of the effective methods is expanding the foggy lane dataset. In this paper, it expands one of the most widely applied datasets, CULane, into FoggyCULane, by artificially generating foggy lane images based on sunny-scene lane images in CULane. This work generates lane images under three different concentrations of fog according to the single image depth prediction and atmospheric scattering model. The foggy lane images are then combined with the lane label of the original images under sunny condition, thus it artificially establishes a new lane dataset named FoggyCULane, containing 107 451 labeled foggy lane images, which is 1.8 times larger than the original CULane dataset. Subsequently, the original CULane dataset and the FoggyCULane dataset are used to train the spatial CNN（SCNN） lane detection neural network, and the training results are evaluated in 12 types of complex lane scenes, which contains 3 different concentrations of foggy scenes, to verify the effectiveness of this method. The results show that compared with the original CULane, FoggyCULane can significantly increase the detection accuracy of lanes in foggy scene, which the detection accuracy increasing from 74.65% to 86.65% under thin mist condition, from 51.41% to 81.53% under moderate fog condition, and from 11.09% to 70.41% under dense fog condition.

Key words: lane detection, deep learning, foggy image generation, expansion of CULane dataset, FoggyCULane, spatial convolutional neural network（SCNN）

摘要： 为了提高深度学习算法在雾天场景下的车道线识别率，扩充雾天车道线数据集是有效途径之一。以目前最具有权威性的CULane数据集为基础，通过对该数据集内晴天车道线图片进行单幅图像深度提取，随后依照大气散射模型生成3种不同浓度的雾天车道线图片，并保留原图的车道线标签，以此方法实现对CULane数据集的人为扩充。通过增加了107?451张带标签的雾天车道线图像，从而将原始CULane数据集扩充了1.8倍，建立了包含雾天图像的新车道线数据集FoggyCULane。分别采用原始CULane数据集和FoggyCULane数据集对SCNN车道识别网络进行训练，并将训练结果在包含3种不同浓度雾天场景的12种复杂车道线场景中进行测试评估，以验证该方法的有效性。研究结果表明，人工生成雾天场景车道线图片以扩充数据集的方法能够在薄雾情况下将雾天车道线的识别率从74.65%提升至86.65%，在中度雾下从51.41%提升至81.53%，在浓雾下从11.09%提升至70.41%。

关键词: 车道线检测, 深度学习, 雾天图片生成, CULane数据集扩充, FoggyCULane, 空间卷积神经网络（SCNN）

XU Zhejun, ZHANG Wei, GUO Hao, ZHANG Yang, LI Qing, DONG Xue. Lane Detection Under Foggy Conditions via FoggyCULane Dataset[J]. Computer Engineering and Applications, 2022, 58(14): 227-235.

徐哲钧, 张暐, 郭昊, 张洋, 李庆, 董雪. 雾天车道线识别方法：FoggyCULane数据集的创建[J]. 计算机工程与应用, 2022, 58(14): 227-235.

References

[1] KIM J，LEE M.Robust lane detection based on convolutional neural network and random sample consensus[C]//International Conference on Neural Information Processing.Cham：Springer，2014：454-461.
[2] LI J，MEI X，PROKHOROV D，et al.Deep neural network for structural prediction and lane detection in traffic scene[J].IEEE Transactions on Neural Networks and Learning Systems，2016，28（3）：690-703.
[3] HUVAL B，WANG T，TANDON S，et al.An empirical evaluation of deep learning on highway driving[J].arXiv：1504.01716，2015.
[4] ZHOU K.TuSimple dataset[DB/OL].（2018-10-20）[2020-10-12].https：//github.com/TuSimple/tusimple-benchmark/issues/3.
[5] CORDTS M，OMRAN M，RAMOS S，et al.The cityscapes dataset for semantic urban scene understanding[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2016：3213-3223.
[6] GEIGER A，LENZ P，STILLER C，et al.Vision meets robotics：the kitti dataset[J].The International Journal of Robotics Research，2013，32（11）：1231-1237.
[7] PAN X，SHI J，LUO P，et al.Spatial as deep：spatial CNN for traffic scene understanding[C]//Proceedings of the AAAI Conference on Artificial Intelligence，2018.
[8] TAREL J P，HAUTIERE N，CORD A，et al.Improved visibility of road scene images under heterogeneous fog[C]//2010 IEEE Intelligent Vehicles Symposium，2010：478-485.
[9] TAREL J P，HAUTIERE N，CARAFFA L，et al.Vision enhancement in homogeneous and heterogeneous fog[J].IEEE Intelligent Transportation Systems Magazine，2012，4（2）：6-20.
[10] JEONG K M，SONG B C.Image synthesis algorithm for road object detection in rainy weather[J].IEIE Transactions on Smart Processing & Computing，2018，7（5）：342-349.
[11] LIU T，CHEN Z，YANG Y，et al.Lane detection in low-light conditions using an efficient data enhancement：light conditions style transfer[C]//2020 IEEE Intelligent Vehicles Symposium（IV），2020：1394-1399.
[12] SAKARIDIS C，DAI D，VAN GOOL L.Semantic foggy scene understanding with synthetic data[J].International Journal of Computer Vision，2018，126（9）：973-992.
[13] KOSCHMIEDER H.Theorie der horizontalen sichtweite：kontrast und sichtweite[J].Keim & Nemnich，1924，12：33-55.
[14] MCCARTNEY E J.Optics of the atmosphere：scattering by molecules and particles[J].Physics Today，1977，30（5）：76.
[15] NAYAR S K，NARASIMHAN S G.Vision in bad weather[C]//Proceedings of the Seventh IEEE International Conference on Computer Vision，1999：820-827.
[16] SMITH M W，CARRIVICK J L，QUINCEY D J.Structure from motion photogrammetry in physical geography[J].Progress in Physical Geography，2016，40（2）：247-275.
[17] DHOND U R，AGGARWAL J K.Structure from stereo-a review[J].IEEE Transactions on Systems，Man，and Cybernetics，1989，19（6）：1489-1510.
[18] DELLAERT F，SEITZ S M，THORPE C E，et al.Structure from motion without correspondence[C]//Proceedings IEEE Conference on Computer Vision and Pattern Recognition，2000：557-564.
[19] ZHANG R，TSAI P S，CRYER J E，et al.Shape-from-shading：a survey[J].IEEE Transactions on Pattern Analysis and Machine Intelligence，1999，21（8）：690-706.
[20] EIGEN D，PUHRSCH C，FERGUS R.Depth map prediction from a single image using a multi-scale deep network[C]//Advances in Neural Information Processing Systems，2014：2366-2374.
[21] GODARD C，MAC AODHA O，FIRMAN M，et al.Digging into self-supervised monocular depth estimation[C]//Proceedings of the IEEE International Conference on Computer Vision，2019：3828-3838.