Multi-Modal Meteorological Forecasting Based on Transformer

doi:10.3778/j.issn.1002-8331.2208-0486

Abstract

Abstract: Thanks to the rapid development of meteorological observation technology, the meteorological industry has accumulated massive meteorological data, which provides an opportunity to build new data-driven meteorological forecasting methods. Due to the long-term dependence and large-scale spatial correlation hidden in meteorological data, and due to the complex coupling relationship between different modalities, meteorological forecasting with deep learning is still a challenging research topic. This paper presents a deep learning model for meteorological forecasting based on multi-modal fusion, using sequential multi-modal data in same atmospheric pressure levels composed of four classical meteorological elements：temperature, relative humidity, U-component of wind and V-component of wind. Specifically, convolutional network is used to learn features from every modality, and with those features, the gating mechanism is introduced to multi-modal weighted fusion. Secondly, the attention mechanism is introduced, which replaces the traditional attention mechanism with parallel spatial-temporal axial attention, in order to effectively learn long-term dependencies and large-scale spatial associations. Architecturally, the Transformer encoder-decoder structure is employed as the overall framework. Extensive comparative experiments have been conducted on the regional ERA5 reanalysis dataset, demonstrating that the proposed method is effective and superior in the prediction of temperature, relative humidity and wind.

Key words: meteorological forecasting, multi-modal fusion, Transformer, axial attention

摘要： 随着气象观测技术的快速发展，气象行业积累了海量的气象大数据，为构建新型的数据驱动的气象预测模型提供了机遇。由于气象数据中存在的长时依赖关系和大范围空间关联关系，以及多模态气象要素间存在的复杂跨模态耦合关系，基于深度学习的气象预测是一个具有挑战性的研究课题。针对“温度、相对湿度、纬向风速、经向风速”四种经典气象要素组成的等气压层时序多模态数据，提出了一种基于多模态融合的气象预测深度学习模型。首先采用卷积网络来学习各个模态的特征，并在此基础上引入门控机制实现多模态加权融合；然后引入注意力机制，以并行时空轴向注意力代替传统的注意力机制，从而有效地学习长时依赖关系和大范围空间关联关系。整体结构上，采用了基于Transformer的编码器-解码器结构。在ERA5再分析数据集（子区域）上进行了对比实验，实验结果表明了所提方法在温度、相对湿度、风速等预测任务上的有效性和优越性。

关键词: 气象预测, 多模态融合, Transformer, 轴向注意力

XIANG Deping, ZHANG Pu, XIANG Shiming, PAN Chunhong. Multi-Modal Meteorological Forecasting Based on Transformer[J]. Computer Engineering and Applications, 2023, 59(10): 94-103.

向德萍, 张普, 向世明, 潘春洪. 基于Transformer的多模态气象预测[J]. 计算机工程与应用, 2023, 59(10): 94-103.

References

[1] BAUER P，THORPE A，BRUNET G.The quiet revolution of numerical weather prediction[J].Nature，2015，525（7567）：47-55.
[2] 许小峰.从物理模型到智能分析——降低天气预报不确定性的新探索[J].气象，2018，3（3）：341-350.
XU X F.From physical model to intelligent analysis：a new exploration to reduce the uncertainty of weather forecast[J].Meteorological Monthly，2018，3（3）：341-350.
[3] S?NDERBY C K，ESPEHOLT L，HEEK J，et al.MetNet：a neural weather model for precipitation forecasting[J].arXiv：2003.12140，2020.
[4] RAVURI S，LENC K，WILLSON M，et al.Skilful precipitation nowcasting using deep generative models of radar[J].Nature，2021，597（7878）：672-677.
[5] SHI X，CHEN Z，WANG H，et al.Convolutional LSTM network：a machine learning approach for precipitation nowcasting[C]//Proceedings of the Advances in Neural Information Processing Systems，2015：802-810.
[6] SHI X，GAO Z，LAUSEN L，et al.Deep learning for precipitation nowcasting：a benchmark and a new model[C]//Proceedings of the Advances in Neural Information Processing Systems，2017：5617-5627.
[7] WANG Y，LONG M，WANG J，et al.PredRNN：recurrent neural networks for predictive learning using spatiotemporal LSTMs[C]//Proceedings of the Advances in Neural Information Processing Systems，2017：879-888.
[8] WANG Y，JIANG L，YANG M，et al.Eidetic 3D LSTM：a model for video prediction and beyond[C]//Proceedings of the International Conference on Learning Representations，2019.
[9] WANG Y，ZHANG J，ZHU H，et al.Memory in memory：a predictive neural network for learning higher-order non-stationarity from spatiotemporal dynamics[C]//Proceedings of the 2019 IEEE Conference on Computer Vision and Pattern Recognition，2019：9154-9162.
[10] VASWANI A，SHAZEER N，PARMAR N，et al.Attention is all you need[C]//Proceedings of the Advances in Neural Information Processing Systems，2017：5998-6008.
[11] WEN Q，ZHOU T，ZHANG C，et al.Transformers in time series：a survey[J].arXiv：2202.07125，2022.
[12] CARION N，MASSA F，SYNNAEVE G，et al.End-to-end object detection with transformers[C]//16th European Conference on Computer Vision.Cham：Springer，2020：213-229.
[13] ZHOU H，ZHANG S，PENG J，et al.Informer：beyond efficient transformer for long sequence time-series forecasting[C]//Proceedings of the 35th AAAI Conference on Artificial Intelligence，2021：11106-11115.
[14] LIU Z，LUO S，LI W，et al.Convtransformer：a convolutional transformer network for video frame synthesis[J].arXiv：2011.10185，2020.
[15] HERSBACH H，BELL B，BERRISFORD P，et al.The ERA5 global reanalysis[J].Quarterly Journal of the Royal Meteorological Society，2020，146（730）：1999-2049.
[16] HO J，KALCHBRENNER N，WEISSENBORN D，et al.Axial attention in multidimensional transformers[J].arXiv：1912.12180，2019.
[17] BERTASIUS G，WANG H，TORRESANI L.Is space-time attention all you need for video understanding[J].arXiv：2102.05095，2021.
[18] BA J L，KIROS J R，HINTON G E.Layer normalization[J].arXiv：1607.06450，2016.
[19] WILLIAMS R J，ZIPSER D.A learning algorithm for continually running fully recurrent neural networks[J].Neural Computation，1989，1（2）：270-280.
[20] PARMAR N，VASWANI A，USZKOREIT J，et al.Image transformer[C]//Proceedings of the 35th International Conference on Machine Learning，2018：4055-4064.
[21] SHELHAMER E，LONG J，DARRELL T.Fully convolutional networks for semantic segmentation[J].IEEE Transactions on Pattern Analysis and Machine Intelligence，2017，39（4）：640-651.
[22] RONNEBERGER O，FISCHER P，BROX T.U-net：convolutional networks for biomedical image segmentation[C]//International Conference on Medical Image Computing and Computer-Assisted Intervention.Cham：Springer，2015：234-241.
[23] AYZEL G，HEISTERMANN M，SOROKIN A，et al.All convolutional neural networks for radar-based precipitation nowcasting[J].Procedia Computer Science，2019，150：186-192.
[24] BALLAS N，YAO L，PAL C，et al.Delving deeper into convolutional networks for learning video representations[J].arXiv：1511.06432，2015.
[25] WANG Y，GAO Z，LONG M，et al.PredRNN++：towards a resolution of the deep-in-time dilemma in spatiotemporal predictive learning[C]//Proceedings of the 35th International Conference on Machine Learning，2018：5110-5119.