Parallel I/O Optimization Based on Ocean Circulation Model

doi:10.3778/j.issn.1002-8331.2306-0114

Abstract

Abstract: Extreme weather is getting worse and worse, which has seriously affected human production and life. How to effectively respond to extreme environmental changes in the future is more related to international competition. Therefore, it is very important to develop high-precision and high-resolution earth system models to promote climate change research. With the continuous improvement of mode accuracy and resolution, I/O performance has become a bottleneck restricting the efficiency and scalability of mode computing. The typical ocean circulation model LICOM3 is selected as the research object. Based on the Tianhe HPC system, the I/O process of the model is tested and analyzed in detail. It is found that under the 3K process, the I/O can account for more than half of the overall running time, which is the key factor limiting the calculation performance of the model. Therefore, the MPI-IO parallel optimization scheme for the main output processes is designed and implemented, which significantly improves the mode operation efficiency and basically solves the mode I/O bottleneck. The test results show that the MPI-IO parallel optimization scheme reduces the output time from 48.58% to 2.44% and increases the output bandwidth to 6.21 GB/s, about 14 times that of the original program. In addition, the factors affecting the performance of parallel I/O are further analyzed to provide reference and guidance for related work.

Key words: parallel I/O optimization, ocean circulation model, MPI-IO

摘要： 极端气象日益加剧，严重影响人类的生产生活。如何有效应对未来极端环境变化，更关乎国际竞争。因此，发展高精度高分辨率的地球系统模式推动气候变化研究至关重要。随着模式精度和分辨率的不断提高，I/O性能已成为制约模式计算效率和可扩展性的瓶颈。选取典型的海洋环流模式LICOM3作为研究对象，基于天河HPC系统，详细测试和分析了模式的I/O逻辑过程，发现在3K进程规模下I/O可占整体运行时间一半以上，是限制模式计算性能的关键因素。为此，设计并实现了主要输出过程的MPI-IO并行优化方案，显著提升了模式运行效率，基本解决了模式I/O瓶颈。测试结果表明，MPI-IO并行优化方案将输出时间占比从48.58%降到2.44%，将输出带宽提升到6.21?GB/s，约是原程序的14倍。此外，进一步分析了影响并行I/O性能的因素，以期为相关工作提供参考和指导。

关键词: 并行I/O优化, 海洋环流模式, MPI-IO

LI Yunlong, ZHU Zijie, MENG Xiangfei, ZHU Xiaoqian, LIN Pengfei, LIU Hailong, LI Geng. Parallel I/O Optimization Based on Ocean Circulation Model[J]. Computer Engineering and Applications, 204, 60(17): 263-271.

李云龙, 祝子杰, 孟祥飞, 朱小谦, 林鹏飞, 刘海龙, 李庚. 海洋环流模式的并行I/O优化研究[J]. 计算机工程与应用, 204, 60(17): 263-271.

References

[1] 栾贻花, 俞永强, 郑伟鹏. 全球高分辨率气候系统模式研究进展[J]. 地球科学进展, 2016, 31(3): 258-268.
LUAN Y H, YU Y Q, ZHENG W P. Review of development and application of high resolution global climate system model[J]. Advances in Earth Science, 2016, 31(3): 258-268.
[2] 唐彦丽, 董文杰, 李立娟, 等. CPL7及其在中国地球系统模式中的应用前景[J]. 地球科学进展, 2015, 30(5): 620-625.
TANG Y L, DONG W J, LI L J, et al. CPL7 and its application prospect in the earth system models of China[J]. Advances in Earth Science, 2015, 30(5): 620-625.
[3] 普业, 李立娟. 高分辨地球系统模式的千核应用[J]. 科研信息化技术与应用, 2010(4): 69-75.
PU Y, LI L J. The application of thousands of CPU cores in high resolution earth system models[J]. E-science Technology & Application, 2010(4): 69-75.
[4] 刘海龙, 俞永强, 李薇, 等. LASG/IAP气候系统海洋模式 (LICOM1.0) 参考手册[M]. 北京: 科学出版社, 2004.
LIU H L, YU Y Q, LI W, et al. LASG/IAP climate system ocean model (LICOM 1.0) reference manual[M]. Beijing: Science Press, 2004.
[5] LIU H L, LIN P F, YU Y Q, et al. The baseline evaluation of LASG/IAP climate system ocean model (LICOM) version 2[J]. Acta Meteorologica Sinica, 2012, 26(3): 60-71.
[6] 魏敏, 徐金秀, 王在志. 并行I/O技术在气候数值模式中的应用研究[J]. 计算机技术与发展, 2014(12): 11-15.
WEI M, XU J X, WANG Z Z. Study and application of parallel I/O technology in numerical climate model[J]. Computer Technology and Development, 2014(12): 11-15.
[7] ZOU Y L, XUE W, LIU S S. A case study of large-scale parallel I/O analysis and optimization for numerical weather prediction system[J]. Future Generation Computer Systems, 2014, 37: 378-389.
[8] 杨斌, 王敬宇, 刘卫国, 等. GRAPES区域模式的输入输出分析和优化[J]. 电子技术应用, 2022, 48(1): 39-45.
YANG B, WANG J Y, LIU W G, et al. Input/Output analysis and optimization for GRAPES regional model[J]. Application of Electronic Technique, 2022, 48(1): 39-45.
[9] 黄伟建, 王鑫, 韩院彬. 并行I/O技术在海洋数值模式中的应用研究[J]. 河北工程大学学报 (自然科学版), 2016, 33(1): 86-90.
HUANG W J, WANG X, HAN Y B. Study and application of parallel I/O technology in numerical ocean model[J]. Journal of Hebei University of Engineering (Natural Science Edition), 2016, 33(1): 86-90.
[10] 季旭, 武海平, 邹寅隆, 等. 面向LICOM2的并行I/O优化[C]//第四届中国科学院超级计算机应用大会论文集, 2014: 37-48.
JI X, WU H P ZOU Y L, et al. Parallel I/O optimization for high resolution ocean model LICOM2[C]//Proceedings of the 4th Chinese Academy of Sciences Supercomputer Applications Conference, 2014: 37-48.
[11] LIAO J W, CHEN J K, YAN H, et al. Parallel I/O optimization of FVCOM for finite volume offshore ocean model[C]//Proceedings of the CCF HPC China, 2021: 13-20.
[12] CORBETT P, FEITELSON D, FINEBERG S, et al. Overview of the MPI-IO parallel I/O interface[C]//Proceedings of the 3rd Workshop on IO in Parallel and Distributed Systems, 1995.
[13] LI J, LIAO W K, CHOUDHARY A, et al. Parallel NetCDF: a scientific high-performance I/O interface[C]//Proceedings of the 2003 ACM/IEEE Conference on Supercomputing, 2003: 39.
[14] YANG M Q . Parallel HDF5 tutorial[C]//Proceedings of the 14th IBM System Scientific User Group (ScicomP), 2008.
[15] DENNIS J M, EDWARDS J, LOY R, et al. An application-level parallel I/O library for earth system models[J]. International Journal of High Performance Computing Applications, 2012, 26(1): 43-53.
[16] JIN C, KLASKY S, HODSON S, et al. Adaptive I/O system (ADIOS)[C]//Proceedings of the Cray User’s Group, 2008.
[17] 张留莹, 王鹏飞, 张峰, 等. 海洋环流模式LICOM的GPU实现与优化[J]. 数据与计算发展前沿, 2020, 2(4): 92-104.
ZHANG L Y, WANG P F, ZHANG F, et al. The implementation and optimization of LICOM on GPUs[J]. Frontiers of Data and Computing, 2020, 2(4): 92-104.
[18] WANG P F, JIANG J R, LIN P F, et al. The GPU version of LASG/IAP climate system ocean model version 3 (LICOM3) under the heterogeneous-compute interface for portability (HIP) framework and its large-scale application[J]. Geoscientific Model Development, 2021, 14(5): 2781-2799.
[19] SYSTEM C F. Lustre: a scalable, highperformancefile system[R]. New York: Cluster File System, 2012.
[20] KUNKEL J, MORRONE C, CHAARAWI M, et al. IOR manual[EB/OL]. (2018-12-31)[2023-06-07]. https://ior.readthedocs.io/en/latest.