Positioning Method of Wind Turbine Blade Tips Based on UAV Vision

doi:10.3778/j.issn.1002-8331.2203-0336

Abstract

Abstract: When applying UAVs for autonomous inspection of wind turbines, the blade tips need to be accurately located. A method for locating the blade tips based on UAV vision is proposed. Firstly, the UAV uses the rough yaw information of the wind turbine to reach the designated position, and uses the Yolov5 model to extract features of the wind turbine blades, hubs and other parts. Secondly, the homography matrix is calculated and decomposed according to the extracted feature points to obtain the UAV offset. Then, a hybrid vision servo-based design corrector is used to correct the position and attitude of UAV, so that the UAV is squared up to the wind turbine and a panoramic image is acquired, while its stability is analyzed. Finally, the blade tips are positioned in 3D space using the panorama and the position and attitude of the UAV. Simulation tests show that the designed controller can enable the UAV to autonomously complete the operation of squaring up to wind turbine and collect a panoramic view within 15 seconds. The proposed method can accurately locate the blade tip of the wind turbine with a positioning error of only within ±5 meters.

Key words: wind turbine, UAV, Yolov5, visual servo, homography matrix, blade tip positioning

摘要： 应用无人机对风力发电机进行自主巡检时，需对其桨叶叶尖做出精准定位。为此提出一种基于无人机视觉的叶尖定位方法。无人机利用风机粗略偏航信息到达指定位置后，并利用Yolov5模型对风机桨叶、轮毂等部位进行特征提取；根据提取到的特征点计算单应性矩阵并将其分解，得到无人机偏移量；基于混合式视觉伺服设计修正器对无人机位姿进行修正，从而使无人机正对风机并进行全景图采集，同时对其进行了稳定性分析；利用全景图以及无人机的位姿对桨叶叶尖进行三维空间定位。仿真实验表明，设计的控制器可以使无人机在15?s内自主完成正对风机操作并对其进行全景图采集，提出的方法可对风机桨叶叶尖进行精准定位，定位误差仅在±5?m内。

关键词: 风力发电机, 无人机, Yolov5, 视觉伺服, 单应性矩阵, 叶尖定位

JIAO Songming, BAI Jianpeng, SHOU Yunfeng, CHEN Yuxi, ZHONG Yufei. Positioning Method of Wind Turbine Blade Tips Based on UAV Vision[J]. Computer Engineering and Applications, 2023, 59(12): 301-308.

焦嵩鸣, 白健鹏, 首云锋, 陈雨溪, 钟宇飞. 基于无人机视觉的风机桨叶叶尖定位方法[J]. 计算机工程与应用, 2023, 59(12): 301-308.

References

[1] 管宏业.最大能源消费国之辨[N].中国商报，2010-07-30（A03）.
GUAN H Y.Identification of the largest energy consuming countries[N].China Business Herald，2010-07-30（A03）.
[2] GAO Z，TANG C，ZHOU X，et al.An overview on development of wind power generation[C]//Proceedings of the 28th Chinese Control and Decision Conference，2016：435-439.
[3] 苏伟.能源系统应“三步走”实现“双碳”[N].中国电力报，2021-11-30.
SU W.The energy system should take “three steps” to realize “carbon peak and neutrality”[N].China Electric Power News，2021-11-30.
[4] CHATTERJEE J，DETHLEFS N.Scientometric review of artificial intelligence for operations & maintenance of wind turbines：the past，present and future[J].Renewable & Sustainable Energy Reviews，2021，144：111051.
[5] 赵春溢，郭洪涛，郭涛，等.一种风机叶片图像采集及缺陷检测系统[J].红外技术，2020，42（12）：1203-1210.
ZHAO C Y，GUO H T，GUO T，et al.Defect detection system based on UAV images for wind turbine blades[J].Infrared Technology，2020，42（12）：1203-1210.
[6] 潘佳捷.风机叶片的无人机自主巡检系统[D].成都：电子科技大学，2020.
PAN J J.An autonomous inspection system for wind turbine blade based on unmanned aerial vehicle[D].Chengdu：University of Electronic Science and Technology of China，2020.
[7] 朱凯华.面向风机叶片巡检的无人机自动航迹线规划研究[D].北京：北京交通大学，2021.
ZHU K H.Research on automatic path planning of UAV for wind turbine blade inspection[D].Beijing：Beijing Jiaotong University，2021.
[8] 勾月凯，代海涛，董健，等.风力发电机组叶片智能巡检系统研究[C]//第五届中国风电后市场专题研讨会论文集，2018：203-208.
GOU Y K，DAI H T，DONG J，et al.Research on intelligent inspection system of wind turbine blades[C]//Proceedings of the 5th China Wind Power Post Market Symposium，2018：203-208.
[9] KANELLAKIS C，FRESK E，MANSOURI S S，et al.Towards visual inspection of wind turbines：a case of visual data acquisition using autonomous aerial robots[J].IEEE Access，2020，8：181650-181661.
[10] GUO H，CUI Q，WANG J，et al.Detecting and positioning of wind turbine blade tips for UAV-based automatic inspection[C]//Proceedings of the 2019 IEEE International Geoscience and Remote Sensing Symposium，2019：1374-1377.
[11] MOOLAN-FEROZE O，KARACHALIOS K，NIKOLAIDIS D N，et al.Simultaneous drone localisation and wind turbine model fitting during autonomous surface inspection[C]//Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems（IROS），2019：2014-2021.
[12] 李国栋，田国会，王洪君，等.一种景物平面法向已知条件下的单应矩阵快速分解算法[J].控制与决策，2014，29（4）：735-738.
LI G D，TIAN G D，WANG H J，et al.A fast homography decomposition algorithm in the case of known normal to the scene plane[J].Control and Decision，2014，29（4）：735-738.
[13] BEN-ARTZI G.Separable four points fundamental matrix[C]//Proceedings of the IEEE Winter Conference on Applications of Computer Vision（WACV），2021：188-196.
[14] 赵玥，陈奕梅.四旋翼无人飞行器控制算法设计[J].计算机工程与应用，2017，53（21）：49-53.
ZHAO Y，CHEN Y M.Design of four rotor unmanned spacecraft control algorithm[J].Computer Engineering and Applications，2017，53（21）：49-53.
[15] 张锦，屈佩琪，孙程，等.基于改进YOLOv5的安全帽佩戴检测算法[J].计算机应用，2022，42（4）：1292-1300.
ZHANG J，QU P Q，SUN C，et al.Safety helmet wearing detection method based on improved YOLOv5[J].Journal of Computer Applications，2022，42（4）：1292-1300.
[16] 孔筱芳，陈钱，顾国华，等.基于全球定位系统的相机标定方法研究[J].兵工学报，2016，37（12）：2301-2307.
KONG X F，CHEN Q，GU G H，et al.A GPS-based camera calibration method[J].Acta Armamentarii，2016，37（12）：2301-2307.