Physics-informed neural networks with residual/gradient-based adaptive sampling methods for solving partial differential equations with sharp solutions

doi:10.1007/s10483-023-2994-7

Applied Mathematics and Mechanics (English Edition) ›› 2023, Vol. 44 ›› Issue (7): 1069-1084.doi: https://doi.org/10.1007/s10483-023-2994-7

Physics-informed neural networks with residual/gradient-based adaptive sampling methods for solving partial differential equations with sharp solutions

Zhiping MAO¹, Xuhui MENG²

1. School of Mathematical Sciences, Fujian Provincial Key Laboratory of Mathematical Modeling and High-Performance Scientific Computing, Xiamen University, Xiamen 361005, Fujian Province, China;
2. Institute of Interdisciplinary Research for Mathematics and Applied Science, School of Mathematics and Statistics, Huazhong University of Science and Technology, Wuhan 430074, China

收稿日期:2023-01-25 修回日期:2023-03-10 出版日期:2023-07-01 发布日期:2023-07-05
通讯作者: Xuhui MENG, E-mail:xuhui_meng@hust.edu.cn
基金资助:
the National Key R&D Program of China (No. 2022YFA1004504), the National Natural Science Foundation of China (Nos. 12171404 and 12201229), and the Fundamental Research Funds for Central Universities of China (No. 20720210037)

Physics-informed neural networks with residual/gradient-based adaptive sampling methods for solving partial differential equations with sharp solutions

Zhiping MAO¹, Xuhui MENG²

1. School of Mathematical Sciences, Fujian Provincial Key Laboratory of Mathematical Modeling and High-Performance Scientific Computing, Xiamen University, Xiamen 361005, Fujian Province, China;
2. Institute of Interdisciplinary Research for Mathematics and Applied Science, School of Mathematics and Statistics, Huazhong University of Science and Technology, Wuhan 430074, China

Received:2023-01-25 Revised:2023-03-10 Online:2023-07-01 Published:2023-07-05
Contact: Xuhui MENG, E-mail:xuhui_meng@hust.edu.cn
Supported by:
the National Key R&D Program of China (No. 2022YFA1004504), the National Natural Science Foundation of China (Nos. 12171404 and 12201229), and the Fundamental Research Funds for Central Universities of China (No. 20720210037)

摘要/Abstract

摘要： We consider solving the forward and inverse partial differential equations (PDEs) which have sharp solutions with physics-informed neural networks (PINNs) in this work. In particular, to better capture the sharpness of the solution, we propose the adaptive sampling methods (ASMs) based on the residual and the gradient of the solution. We first present a residual only-based ASM denoted by ASM I. In this approach, we first train the neural network using a small number of residual points and divide the computational domain into a certain number of sub-domains, then we add new residual points in the sub-domain which has the largest mean absolute value of the residual, and those points which have the largest absolute values of the residual in this sub-domain as new residual points. We further develop a second type of ASM (denoted by ASM II) based on both the residual and the gradient of the solution due to the fact that only the residual may not be able to efficiently capture the sharpness of the solution. The procedure of ASM II is almost the same as that of ASM I, and we add new residual points which have not only large residuals but also large gradients. To demonstrate the effectiveness of the present methods, we use both ASM I and ASM II to solve a number of PDEs, including the Burger equation, the compressible Euler equation, the Poisson equation over an L-shape domain as well as the high-dimensional Poisson equation. It has been shown from the numerical results that the sharp solutions can be well approximated by using either ASM I or ASM II, and both methods deliver much more accurate solutions than the original PINNs with the same number of residual points. Moreover, the ASM II algorithm has better performance in terms of accuracy, efficiency, and stability compared with the ASM I algorithm. This means that the gradient of the solution improves the stability and efficiency of the adaptive sampling procedure as well as the accuracy of the solution. Furthermore, we also employ the similar adaptive sampling technique for the data points of boundary conditions (BCs) if the sharpness of the solution is near the boundary. The result of the L-shape Poisson problem indicates that the present method can significantly improve the efficiency, stability, and accuracy.

关键词: physics-informed neural network (PINN), adaptive sampling, high-dimension, L-shape Poisson equation, accuracy

Abstract: We consider solving the forward and inverse partial differential equations (PDEs) which have sharp solutions with physics-informed neural networks (PINNs) in this work. In particular, to better capture the sharpness of the solution, we propose the adaptive sampling methods (ASMs) based on the residual and the gradient of the solution. We first present a residual only-based ASM denoted by ASM I. In this approach, we first train the neural network using a small number of residual points and divide the computational domain into a certain number of sub-domains, then we add new residual points in the sub-domain which has the largest mean absolute value of the residual, and those points which have the largest absolute values of the residual in this sub-domain as new residual points. We further develop a second type of ASM (denoted by ASM II) based on both the residual and the gradient of the solution due to the fact that only the residual may not be able to efficiently capture the sharpness of the solution. The procedure of ASM II is almost the same as that of ASM I, and we add new residual points which have not only large residuals but also large gradients. To demonstrate the effectiveness of the present methods, we use both ASM I and ASM II to solve a number of PDEs, including the Burger equation, the compressible Euler equation, the Poisson equation over an L-shape domain as well as the high-dimensional Poisson equation. It has been shown from the numerical results that the sharp solutions can be well approximated by using either ASM I or ASM II, and both methods deliver much more accurate solutions than the original PINNs with the same number of residual points. Moreover, the ASM II algorithm has better performance in terms of accuracy, efficiency, and stability compared with the ASM I algorithm. This means that the gradient of the solution improves the stability and efficiency of the adaptive sampling procedure as well as the accuracy of the solution. Furthermore, we also employ the similar adaptive sampling technique for the data points of boundary conditions (BCs) if the sharpness of the solution is near the boundary. The result of the L-shape Poisson problem indicates that the present method can significantly improve the efficiency, stability, and accuracy.

Key words: physics-informed neural network (PINN), adaptive sampling, high-dimension, L-shape Poisson equation, accuracy

中图分类号:

74J05
74J25

Zhiping MAO, Xuhui MENG. Physics-informed neural networks with residual/gradient-based adaptive sampling methods for solving partial differential equations with sharp solutions[J]. Applied Mathematics and Mechanics (English Edition), 2023, 44(7): 1069-1084.

参考文献

[1] LECUN, Y., BENGIO, Y., and HINTON, G. Deep learning. nature, 521(7553), 436-444(2015)
[2] MIKOLOV, T., DEORAS, A., POVEY, D., BURGET, L., and ČERNOCKỲ, J. Strategies for training large scale neural network language models. 2011 IEEE Workshop on Automatic Speech Recognition & Understanding, Hawaii, 196-201(2011)
[3] HINTON, G., DENG, L., YU, D., DAHL, G., MOHAMED, A. R., JAITLY, N., SENIOR, A., VANHOUCKE, V., NGUYEN, P., SAINATH, T. N., and KINGSBURY, B. Deep neural networks for acoustic modeling in speech recognition. IEEE Signal Processing Magazine, 29(6), 82-97(2012)
[4] SAINATH, T. N., MOHAMED, A. R., KINGSBURY, B., and RAMABHADRAN, B. Deep convolutional neural networks for LVCSR. 2013 IEEE International Conference on Acoustics, Speech and Signal Processing, Canada, 8614-8618(2013)
[5] KRIZHEVSKY, A., SUTSKEVER, I., and HINTON, G. E. Imagenet classification with deep convolutional neural networks. Communications of the ACM, 60(6), 84-90(2017)
[6] TOMPSON, J. J., JAIN, A., LECUN, Y., and BREGLER, C. Joint training of a convolutional network and a graphical model for human pose estimation. Advances in Neural Information Processing Systems, 271799-1807(2014)
[7] RAISSI, M., PERDIKARIS, P., and KARNIADAKIS, G. E. Physics-informed neural networks:a deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. Journal of Computational Physics, 378, 686-707(2019)
[8] WEINAN, E. and YU, B. The deep ritz method:a deep learning-based numerical algorithm for solving variational problems. Communications in Mathematics and Statistics, 6(1), 1-12(2018)
[9] HAN, J., JENTZEN, A., and WEINAN, E. Solving high-dimensional partial differential equations using deep learning. Proceedings of the National Academy of Sciences, 115(34), 8505-8510(2018)
[10] LONG, Z., LU, Y., MA, X., and DONG, B. PDE-Net:learning PDEs from data. International Conference on Machine Learning, 80, 3208-3216(2018)
[11] LONG, Z., LU, Y., and DONG, B. PDE-Net 2.0:learning PDEs from data with a numericsymbolic hybrid deep network. Journal of Computational Physics, 399, 108925(2019)
[12] SIRIGNANO, J. and SPILIOPOULOS, K. DGM:a deep learning algorithm for solving partial differential equations. Journal of computational physics, 375, 1339-1364(2018)
[13] PANG, G., LU, L., and KARNIADAKIS, G. E. FPINNs:fractional physics-informed neural networks. SIAM Journal on Scientific Computing, 41(4), 2603-2626(2019)
[14] MENG, X. and KARNIADAKIS, G. E. A composite neural network that learns from multifidelity data:application to function approximation and inverse PDE problems. Journal of Computational Physics, 401, 109020(2020)
[15] ZHANG, D., LU, L., GUO, L., and KARNIADAKIS, G. E. Quantifying total uncertainty in physics-informed neural networks for solving forward and inverse stochastic problems. Journal of Computational Physics, 397, 108850(2019)
[16] LU, L., MENG, X., MAO, Z., and KARNIADAKIS, G. E. DeepXDE:a deep learning library for solving differential equations. SIAM Review, 63(1), 208-228(2021)
[17] RAISSI, M., YAZDANI, A., and KARNIADAKIS, G. E. Hidden fluid mechanics:a Navier-Stokes informed deep learning framework for assimilating flow visualization data. arXiv Preprint, arXiv:1808.04327(2018) https://doi.org/10.48550/arXiv.1808.04327
[18] RAISSI, M., WANG, Z., TRIANTAFYLLOU, M. S., and KARNIADAKIS, G. E. Deep learning of vortex-induced vibrations. Journal of Fluid Mechanics, 861, 119-137(2019)
[19] CAI, S., MAO, Z., WANG, Z., YIN, M., and KARNIADAKIS, G. E. Physics-informed neural networks (PINNs) for fluid mechanics:a review. Acta Mechanica Sinica, 37(12), 1727-1738(2021)
[20] MAO, Z., JAGTAP, A. D., and KARNIADAKIS, G. E. Physics-informed neural networks for high-speed flows. Computer Methods in Applied Mechanics and Engineering, 360, 112789(2020)
[21] JAGTAP, A. D., MAO, Z., ADAMS, N., and KARNIADAKIS, G. E. Physics-informed neural networks for inverse problems in supersonic flows. Journal of Computational Physics, 466, 111402(2022)
[22] BAYDIN, A. G., PEARLMUTTER, B. A., RADUL, A. A., and SISKIND, J. M. Automatic differentiation in machine learning:a survey. Journal of Machine Learning Research, 18, 1-43(2018)
[23] BERGER, M. J. and OLIGER, J. Adaptive mesh refinement for hyperbolic partial differential equations. Journal of Computational Physics, 53(3), 484-512(1984)
[24] BERGER, M. J. and COLELLA, P. Local adaptive mesh refinement for shock hydrodynamics. Journal of Computational Physics, 82(1), 64-84(1989)
[25] VERFÜRTH, R. A posteriori error estimation and adaptive mesh-refinement techniques. Journal of Computational and Applied Mathematics, 50(1-3), 67-83(1994)
[26] BAEZA, A. and MULET, P. Adaptive mesh refinement techniques for high-order shock capturing schemes for multi-dimensional hydrodynamic simulations. International Journal for Numerical Methods in Fluids, 52(4), 455-471(2006)
[27] YU, J., LU, L., MENG, X., and KARNIADAKIS, G. E. Gradient-enhanced physics-informed neural networks for forward and inverse pde problems. Computer Methods in Applied Mechanics and Engineering, 393, 114823(2022)
[28] WU, C., ZHU, M., TAN, Q., KARTHA, Y., and LU, L. A comprehensive study of nonadaptive and residual-based adaptive sampling for physics-informed neural networks. Computer Methods in Applied Mechanics and Engineering, 403, 115671(2023)
[29] GAO, Z., YAN, L., and ZHOU, T. Failure-informed adaptive sampling for PINNs. arXiv Preprint, arXiv:2210.00279(2022) https://doi.org/10.48550/arXiv.2210.00279
[30] GUO, J., WANG, H., and HOU, C. A novel adaptive causal sampling method for physicsinformed neural networks. arXiv Preprint, arXiv:2210.12914(2022) https://doi.org/10.48550/arXiv.2210.12914
[31] HAN, J., CAI, Z., WU, Z., and ZHOU, X. Residual-quantile adjustment for adaptive training of physics-informed neural network. arXiv Preprint, arXiv:2209.05315(2022) https://doi.org/10.48550/arXiv.2209.05315

Physics-informed neural networks with residual/gradient-based adaptive sampling methods for solving partial differential equations with sharp solutions

Physics-informed neural networks with residual/gradient-based adaptive sampling methods for solving partial differential equations with sharp solutions

RichHTML

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 15

编辑推荐

Metrics

本文评价

[1]	J. N. FUHG, A. KARMARKAR, T. KADEETHUM, H. YOON, N. BOUKLAS. Deep convolutional Ritz method: parametric PDE surrogates without labeled data[J]. Applied Mathematics and Mechanics (English Edition), 2023, 44(7): 1151-1174.
[2]	Xuhui MENG. Variational inference in neural functional prior using normalizing flows: application to differential equation and operator learning problems[J]. Applied Mathematics and Mechanics (English Edition), 2023, 44(7): 1111-1124.
[3]	Yichuan HE, Zhicheng WANG, Hui XIANG, Xiaomo JIANG, Dawei TANG. An artificial viscosity augmented physics-informed neural network for incompressible flow[J]. Applied Mathematics and Mechanics (English Edition), 2023, 44(7): 1101-1110.
[4]	W. WU, M. DANEKER, M. A. JOLLEY, K. T. TURNER, L. LU. Effective data sampling strategies and boundary condition constraints of physics-informed neural networks for identifying material properties in solid mechanics[J]. Applied Mathematics and Mechanics (English Edition), 2023, 44(7): 1039-1068.
[5]	Jizeng WANG, Yonggu FENG, Cong XU, Xiaojing LIU, Youhe ZHOU. Multiresolution method for bending of plates with complex shapes[J]. Applied Mathematics and Mechanics (English Edition), 2023, 44(4): 561-582.
[6]	Ziqiang CHENG, Shihao LIU, Yan JIANG, Jianfang LU, Mengping ZHANG, Shuhai ZHANG. A high order boundary scheme to simulate complex moving rigid body under impingement of shock wave[J]. Applied Mathematics and Mechanics (English Edition), 2021, 42(6): 841-854.
[7]	Qiang LI, Yu GU, Huatao WANG. The influence of temperature on flow-induced forces on quartzcrystal-microbalance sensors in a Chinese liquor identification electronic-nose: three-dimensional computational fluid dynamics simulation and analysis[J]. Applied Mathematics and Mechanics (English Edition), 2019, 40(9): 1301-1312.
[8]	Lin ZHOU, Zhenghong GAO, Yuan GAO. An approach for choosing discretization schemes and grid size based on the convection-diffusion equation[J]. Applied Mathematics and Mechanics (English Edition), 2018, 39(6): 877-890.
[9]	Yunlong LI, Wei CAO. Research of influence of reduced-order boundary on accuracy and solution of interior points[J]. Applied Mathematics and Mechanics (English Edition), 2017, 38(1): 111-124.
[10]	姜振华;阎超;于剑. High-order discontinuous Galerkin solver on hybrid anisotropic meshes for laminar and turbulent simulations[J]. Applied Mathematics and Mechanics (English Edition), 2014, 35(7): 799-812.
[11]	P. AKBARZADEH. Cavitating/non-cavitating flows simulation by third-order finite volume scheme and power-law preconditioning method[J]. Applied Mathematics and Mechanics (English Edition), 2013, 34(2): 209-228.
[12]	宫兆新;鲁传敬;黄华雄. Effect of regularized delta function on accuracy of immersed boundary method[J]. Applied Mathematics and Mechanics (English Edition), 2012, 33(11): 1453-1466.
[13]	宫兆新鲁传敬黄华雄. Accuracy analysis of immersed boundary method using method of manufactured solutions[J]. Applied Mathematics and Mechanics (English Edition), 2010, 31(10): 1197-1208.
[14]	唐旭海郑超吴圣川张建海. A novel four-node quadrilateral element with continuous nodal stress[J]. Applied Mathematics and Mechanics (English Edition), 2009, 30(12): 1519-.
[15]	谢小平;周天孝. COARSE-MESH-ACCURACY IMPROVEMENT OF BILINEAR Q₄-PLANE ELEMENT BY THE COMBINED HYBRID FINITE ELEMENT METHOD[J]. Applied Mathematics and Mechanics (English Edition), 2003, 24(12): 1456-1465.