基于增广拉格朗日法的动态平衡物理信息神经网络

doi:10.3778/j.issn.1002-8331.2502-0178

摘要/Abstract

摘要： 物理信息神经网络（physics-informed neural network，PINN）是深度学习在求解偏微分方程（partial differential equation，PDE）领域中的一个重要方法。该方法通过将PDE的物理约束直接嵌入神经网络的损失函数，有效缓解了传统方法对大量数据的依赖问题。然而，在标准PINN框架中，各损失项（如PDE约束、初始条件和边界条件等）的权重通常设定为固定值，缺乏随训练过程动态调整的能力，导致难以平衡各损失项的影响，从而影响预测精度。针对上述问题，提出了一种基于增广拉格朗日法（augmented Lagrangian method，ALM）的动态平衡物理信息神经网络。该方法通过引入自适应的拉格朗日乘子和惩罚参数动态更新规则，增强了对不同约束条件的灵活处理能力与适应性。最后，通过在Burgers方程、Helmholtz方程等五个典型方程上的数值实验验证，结果表明，与现有的动态平衡策略相比，该方法在预测精度上取得了显著提升。

关键词: 偏微分方程（PDE）求解, 物理信息神经网络（PINN）, 自适应加权, 增广拉格朗日法（ALM）, 深度学习

Abstract: Physics-informed neural network （PINN） represents a significant approach within deep learning for solving partial differential equations （PDE）. This approach significantly reduces the traditional dependence on large datasets by directly incorporating the physical constraints of PDE into the neural network’s loss function. However, in the standard PINN framework, the weights of various loss terms （such as PDE constraints, initial conditions, and boundary conditions） are typically assigned fixed values and lack the capability to adjust dynamically during the training process. This limitation makes it challenging to balance the influence of each loss term, thereby potentially compromising prediction accuracy. To address this issue, a dynamically balanced physics-informed neural network based on the augmented Lagrangian method （ALM） is proposed. This approach introduces dynamic update rules for adaptive Lagrange multipliers and penalty parameters, thereby enhancing the model’s flexibility in handling various constraint conditions and improving its adaptability. Finally, numerical experiments are conducted on five typical equations, including the Burgers equation and the Helmholtz equation. The results show that, compared with the existing dynamic balanced strategy, this method has achieved a significant improvement in prediction accuracy.

Key words: partial differential equation （PDE） solving, physics-informed neural network （PINN）, adaptive weighting, augmented Lagrangian method （ALM）, deep learning

童剑城, 范斌, 林至诚, 熊美馨, 肖瑶. 基于增广拉格朗日法的动态平衡物理信息神经网络[J]. 计算机工程与应用, 2025, 61(20): 170-181.

TONG Jiancheng, FAN Bin, LIN Zhicheng, XIONG Meixin, XIAO Yao. Dynamic Balanced Physics-Informed Neural Network Based on Augmented Lagrange Method[J]. Computer Engineering and Applications, 2025, 61(20): 170-181.

参考文献

[1] AMES W F. Numerical methods for partial differential equations[M]. 3rd ed. Boston: Academic Press, 1992.
[2] LU Y, LU J. A universal approximation theor- em of deep neural networks for expressing probability distributions[C]//Advances in Neural Information Processing Systems, 2020: 3094-3105.
[3] HORNIK K, STINCHCOMBE M, WHITE H. Universal appro-ximation of an unknown mapping and its derivatives using multilayer feedforward networks[J]. Neural Networks, 1990, 3(5): 551-560.
[4] RAISSI M, PERDIKARIS P, KARNIADAKIS G E. Physics-informed neural networks: a deep learning framework for sol-ving forward and inverse problems involving nonlinear partial differential equations[J]. Journal of Computational Physics, 2019, 378: 686-707.
[5] RALL L B. Automatic differentiation: techniques and applications[M]. Berlin: Springer-Verlag, 1981.
[6] JAGTAP A D, KAWAGUCHI K, KARNIADAKIS G E. Adaptive activation functions accelerate convergence in deep and physics-informed neural networks[J]. Journal of Computational Physics, 2020, 404: 109136.
[7] JAGTAP A D, KAWAGUCHI K, EM KARNIADAKIS G. Locally adaptive activation functions with slope recovery for deep and physics-informed neural networks[J]. Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences, 2020, 476: 20200334.
[8] JIN X W, CAI S Z, LI H, et al. NSFnets (Navier-Stokes flow nets): physics-informed neural networks for the incompressible Navier-Stokes equations[J]. Journal of Computational Physics, 2021, 426: 109951.
[9] ZHENG Q, ZENG L Z, KARNIADAKIS G E. Physics-informed semantic inpainting: application to geostatistical modeling[J]. Journal of Computational Physics, 2020, 419: 109676.
[10] XIAO L S, LI P, SUN F L, et al. Development and validation of a deep learning-based model using computed tomography imaging for predicting disease severity of coronavirus disease 2019[J]. Frontiers in Bioengineering and Biotechnology, 2020, 8: 898.
[11] ANAGNOSTOPOULOS S J, TOSCANO J D, STERGIOPULOS N, et al. Residual-based attention in physics-informed neural networks[J]. Computer Methods in Applied Mechanics and Engineering, 2024, 421: 116805.
[12] WIGHT C L, ZHAO J. Solving Allen-cahn and cahn-Hilliard equations using the adaptive physics informed neural networks[J] arXiv:2007.04542, 2020.
[13] 邓书超, 宋孝天, 钟旻霄, 等. 一种求解偏微分方程的动态平衡物理信息神经网络[J]. 中国科学: 信息科学, 2024, 54(8): 1843-1859.
DENG S C, SONG X T, ZHONG M X, et al. A dynamic balanced physical information neural network for solving partial differential equations[J]. Scientia Sinica: Informationis, 2024, 54(8): 1843-1859.
[14] WANG S F, TENG Y J, PERDIKARIS P. Understanding and mitigating gradient flow pathologies in physics-informed neural networks[J]. SIAM Journal on Scientific Computing, 2021, 43(5): 3055-3081.
[15] MADDU S, STURM D, MüLLER C L, et al. Inverse Dirichlet weighting enables reliable training of physics informed neural networks[J]. Machine Learning: Science and Technology, 2022, 3(1): 015026.
[16] VEMURI S K, DENZLER J. Gradient statistics-based multi-objective optimization in physics-informed neural networks[J]. Sensors, 2023, 23(21): 8665.
[17] WANG S F, YU X L, PERDIKARIS P. When and why PINNs fail to train: a neural tangent kernel perspective[J]. Journal of Computational Physics, 2022, 449: 110768.
[18] XIANG Z X, PENG W, LIU X, et al. Self-adaptive loss balanced physics-informed neural networks[J]. Neurocomputing, 2022, 496: 11-34.
[19] MCCLENNY L D, BRAGA-NETO U M. Self-adaptive physics-informed neural networks[J]. Journal of Computational Physics, 2023, 474: 111722.
[20] BASIR S, SENOCAK I. Physics and equality constrained artificial neural networks: application to forward and inverse problems with multi-fidelity data fusion[J]. Journal of Computational Physics, 2022, 463: 111301.
[21] BASIR S, SENOCAK I. An adaptive augmented Lagrangian method for training physics and equality constrained artificial neural networks[J]. arXiv:2306.04904, 2023.
[22] SON H, CHO S W, HWANG H J. Enhanced physics-informed neural networks with augmented Lagrangian relaxation met-hod (AL-PINNs)[J]. Neurocomputing, 2023, 548: 126424.
[23] WANG Y, YAO Y Z, GUO J W, et al. A practical PINN framework for multi-scale problems with multi-magnitude loss terms[J]. Journal of Computational Physics, 2024, 510: 113112.
[24] 李野, 陈松灿. 基于物理信息的神经网络: 最新进展与展望[J]. 计算机科学, 2022, 49(4): 254-262.
LI Y, CHEN S C. Physics-informed neural networks: recent advances and prospects[J]. Computer Science, 2022, 49(4): 254-262.
[25] BOTTOU L. Large-scale machine learning with stochastic gradient descent[C]//Proceedings of the 19th International Con-ference on Computational Statistics Paris France(COMPSTAT’2010). Cham: Springer, 2010: 177-186.
[26] KINGA D, ADAM J B. A method for stocha-stic optimization[C]//Proceedings of the International Conference on Learning Representations (ICLR), 2015.
[27] BYRD R H, LU P H, NOCEDAL J, et al. A limited memory algorithm for bound constrained optimization[J]. SIAM Journal on Scientific Computing, 1995, 16(5): 1190-1208.
[28] BERTSEKAS D P. Multiplier methods: a survey[J]. Automatica, 1976, 12(2): 133-145.
[29] NANDWANI Y, PATHAK A, SINGLA P. A primal dual formulation for deep learning with constraints[C]//Advances in Neural Information Processing Systems, 2019: 12157-12168.
[30] SANGALLI S, ERDIL E, H?TKER A, et al. Constrained optimization to train neural network- s on critical and under-represented classes[C]//Advances in Neural Information Processing Systems, 2021: 25400-25411.
[31] FIORETTO F, VAN HENTENRYCK P, MAK T W K, et al. Lagrangian duality for constrained deep learning[C]//Proceedings of the European Conference on Machine Learning and Knowledge Discovery in Databases. Cham: Springer, 2021: 118-135.
[32] GLOROT X, BENGIO Y. Understanding the difficulty of tra-ining deep feedforward neural ne-tworks[C]//Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics, 2010: 249-256.
[33] BASDEVANT C, DEVILLE M, HALDENWANG P, et al. Spectral and finite difference solutions of the Burgers equation[J]. Computers & Fluids, 1986, 14(1): 23-41.
[34] LI J, FENG Z, SCHUSTER G. Wave-equation dispersion inversion[J]. Geophysical Journal International, 2017, 208(3): 1567-1578.