Neural network as a function approximator and its application in solving differential equations

doi:10.1007/s10483-019-2429-8

Applied Mathematics and Mechanics (English Edition) ›› 2019, Vol. 40 ›› Issue (2): 237-248.doi: https://doi.org/10.1007/s10483-019-2429-8

Neural network as a function approximator and its application in solving differential equations

Zeyu LIU^1,2, Yantao YANG^1,2, Qingdong CAI^1,2,3

1. State Key Laboratory for Turbulence and Complex System, Peking University, Beijing 100871, China;
2. Department of Mechanics and Engineering Science, College of Engineering, Peking University, Beijing 100871, China;
3. Center for Applied Physics and Technology, College of Engineering, Peking University, Beijing 100871, China

收稿日期:2018-09-05 修回日期:2018-10-25 出版日期:2019-02-01 发布日期:2019-02-01
通讯作者: Qingdong CAI E-mail:caiqd@pku.edu.cn
基金资助:
Project supported by the National Natural Science Foundation of China (No. 11521091)

Neural network as a function approximator and its application in solving differential equations

Zeyu LIU^1,2, Yantao YANG^1,2, Qingdong CAI^1,2,3

1. State Key Laboratory for Turbulence and Complex System, Peking University, Beijing 100871, China;
2. Department of Mechanics and Engineering Science, College of Engineering, Peking University, Beijing 100871, China;
3. Center for Applied Physics and Technology, College of Engineering, Peking University, Beijing 100871, China

Received:2018-09-05 Revised:2018-10-25 Online:2019-02-01 Published:2019-02-01
Contact: Qingdong CAI E-mail:caiqd@pku.edu.cn
Supported by:
Project supported by the National Natural Science Foundation of China (No. 11521091)

摘要/Abstract

摘要： A neural network (NN) is a powerful tool for approximating bounded continuous functions in machine learning. The NN provides a framework for numerically solving ordinary differential equations (ODEs) and partial differential equations (PDEs) combined with the automatic differentiation (AD) technique. In this work, we explore the use of NN for the function approximation and propose a universal solver for ODEs and PDEs. The solver is tested for initial value problems and boundary value problems of ODEs, and the results exhibit high accuracy for not only the unknown functions but also their derivatives. The same strategy can be used to construct a PDE solver based on collocation points instead of a mesh, which is tested with the Burgers equation and the heat equation (i.e., the Laplace equation).

关键词: stress structure, singularity, Z₁ region, stress triaxiality Rσ, CTOD, shear lip, interpolative method, degree parameter of the plane strain state, ordinary differential equation (ODE) solver, neural network (NN), partial differential equation (PDE) solver, function approximation

Abstract: A neural network (NN) is a powerful tool for approximating bounded continuous functions in machine learning. The NN provides a framework for numerically solving ordinary differential equations (ODEs) and partial differential equations (PDEs) combined with the automatic differentiation (AD) technique. In this work, we explore the use of NN for the function approximation and propose a universal solver for ODEs and PDEs. The solver is tested for initial value problems and boundary value problems of ODEs, and the results exhibit high accuracy for not only the unknown functions but also their derivatives. The same strategy can be used to construct a PDE solver based on collocation points instead of a mesh, which is tested with the Burgers equation and the heat equation (i.e., the Laplace equation).

Key words: stress structure, singularity, Z₁ region, stress triaxiality Rσ, CTOD, shear lip, interpolative method, degree parameter of the plane strain state, partial differential equation (PDE) solver, neural network (NN), function approximation, ordinary differential equation (ODE) solver

中图分类号:

65D15

Zeyu LIU, Yantao YANG, Qingdong CAI. Neural network as a function approximator and its application in solving differential equations[J]. Applied Mathematics and Mechanics (English Edition), 2019, 40(2): 237-248.

参考文献

[1] VAN GERVEN, M. and BOHTE, S. Artificial neural networks as models of neural information processing. Frontiers in Computational Neuroscience, 11, 00114(2017)
[2] HAYKIN, S. S. Neural Networks and Learning Machines, 3rd ed., Prentice Hall, Englewood, New Jersey (2008)
[3] KRIZHEVSKY, A., SUTSKEVER, I., and HINTON, G. E. Imagenet classification with deep convolutional neural networks. Proceedings of the 25th International Conference on Neural Information Processing Systems, 1, 1097-1105(2012)
[4] LECUN, Y., BENGIO, Y., and HINTON, G. Deep learning. nature, 521, 436-444(2015)
[5] LAKE, B. M., SALAKHUTDINOV, R., and TENENBAUM, J. B. Human-level concept learning through probabilistic program induction. Science, 350, 1332-1338(2015)
[6] ALIPANAHI, B., DELONG, A., WEIRAUCH, M. T., and FREY, B. J. Predicting the sequence specificities of DNA-and RNA-binding proteins by deep learning. Nature Biotechnology, 33, 831-838(2015)
[7] MCCULLOCH, W. S. and PITTS, W. A logical calculus of the ideas immanent in nervous activity. The Bulletin of Mathematical Biophysics, 5, 115-133(1943)
[8] ROSENBLATT, F. The perceptron:a probabilistic model for information storage and organization in the brain. Psychological Review, 65, 386-408(1958)
[9] ROSENBLATT, F. Principles of Neurodynamics:Perceptions and the Theory of Brain Mechanism, Spartan Books, Washington, D. C. (1961)
[10] MINSKY, M. and PAPERT, S. A. Perceptrons:An Introduction to Computational Geometry, MIT Press, Cambridge (1969)
[11] WERBOS, P. Beyond Regression:New Tools for Prediction and Analysis in the Behavioral Sciences, Ph. D. dissertation, Harvard University (1974)
[12] RALL, L. B. Automatic Differentiation:Techniques and Applications, Springer-Verlag, Berlin (1981)
[13] BAYDIN, A. G., PEARLMUTTER, B. A., RADUL, A. A., and SISKIND, J. M. Automatic differentiation in machine learning:a survey. The Journal of Machine Learning Research, 18, 1-43(2018)
[14] RAISSI, M., PERDIKARIS, P., and KARNIADAKIS, G. E. Physics informed deep learning (part i):data-driven solutions of nonlinear partial differential equations. arXiv, arXiv:1711.10561(2017) https://arxiv.org/abs/1711.10561
[15] MALL, S. and CHAKRAVERTY, S. Application of legendre neural network for solving ordinary differential equations. Applied Soft Computing, 43, 347-356(2016)
[16] MALL, S. and CHAKRAVERTY, S. Chebyshev neural network based model for solving laneemden type equations. Applied Mathematics and Computation, 247, 100-114(2014)
[17] BERG, J. and NYSTRÖM, K. A unified deep artificial neural network approach to partial differential equations in complex geometries. arXiv, arXiv:1711.06464(2017) https://arxiv.org/abs/1711.06464
[18] MAI-DUY, N. and TRAN-CONG, T. Numerical solution of differential equations using multiquadric radial basis function networks. Neural Networks, 14, 185-199(2001)
[19] JIANYU, L., SIWEI, L., YINGJIAN, Q., and YAPING, H. Numerical solution of elliptic partial differential equation using radial basis function neural networks. Neural Networks, 16, 729-734(2003)
[20] ABADI, M., BARHAM, P., CHEN, J., CHEN, Z., DAVIS, A., DEAN, J., DEVIN, M., GHEMAWAT, S., IRVING, G., ISARD, M., and KUDLUR, M. Tensorflow:a system for large-scale machine learning. The 12th USENIX Symposium on Operating Systems Design and Implementation, 16, 265-283(2016)
[21] HORNIK, K., STINCHCOMBE, M., and WHITE, H. Multilayer feedforward networks are universal approximators. Neural Networks, 2, 359-366(1989)
[22] CYBENKO, G. Approximation by superpositions of a sigmoidal function. Mathematics of Control, Signals and Systems, 2, 303-314(1989)
[23] JONES, L. K. Constructive approximations for neural networks by sigmoidal functions. Proceedings of the IEEE, 78, 1586-1589(1990)
[24] CARROLL, S. and DICKINSON, B. Construction of neural networks using the Radon transform. IEEE International Conference on Neural Networks, 1, 607-611(1989)
[25] LIU, D. C. and NOCEDAL, J. On the limited memory BFGS method for large scale optimization. Mathematical Programming, 45, 503-528(1989)
[26] LEE, C. B. Possible universal transitional scenario in a flat plate boundary layer:measurement and visualization. Physical Review E, 62, 3659-3670(2000)
[27] LEE, C. B. and WU, J. Z. Transition in wall-bounded flows. Applied Mechanics Reviews, 61, 030802(2008)
[28] LEE, C. B. New features of CS solitons and the formation of vortices. Physics Letters A, 247, 397-402(1998)
[29] LEE, C. B. and FU, S. On the formation of the chain of ring-like vortices in a transitional boundary layer. Experiments in Fluids, 30, 354-357(2001)

Neural network as a function approximator and its application in solving differential equations

Neural network as a function approximator and its application in solving differential equations

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 15

编辑推荐

Metrics

本文评价

[1]	Yansong WANG, Baolin WANG, Youjiang CUI, Kaifa WANG. Anti-plane pull-out of a rigid line inclusion from an elastic medium[J]. Applied Mathematics and Mechanics (English Edition), 2023, 44(5): 809-822.
[2]	Wei CHEN, Guozhen WANG, Yiqun LI, Lin WANG, Zhouping YIN. The quaternion beam model for hard-magnetic flexible cantilevers[J]. Applied Mathematics and Mechanics (English Edition), 2023, 44(5): 787-808.
[3]	Jian CHEN, Yawei WANG, Xianfang LI. Antiplane shear crack in a prestressed elastic medium based on the couple stress theory[J]. Applied Mathematics and Mechanics (English Edition), 2023, 44(4): 583-602.
[4]	Kuan LU, Yushu CHEN, Lei HOU. Bifurcation characteristics analysis of a class of nonlinear dynamical systems based on singularity theory[J]. Applied Mathematics and Mechanics (English Edition), 2017, 38(9): 1233-1246.
[5]	Z. ZHURAVLOVA. Stress analysis near the tips of a transverse crack in an elastic semi-strip[J]. Applied Mathematics and Mechanics (English Edition), 2017, 38(7): 935-956.
[6]	Dan WANG, Yushu CHEN, M. WIERCIGROCH, Qingjie CAO. Bifurcation and dynamic response analysis of rotating blade excited by upstream vortices[J]. Applied Mathematics and Mechanics (English Edition), 2016, 37(9): 1251-1274.
[7]	Xiaodong WANG, Yushu CHEN, Lei HOU. Nonlinear dynamic singularity analysis of two interconnected synchronous generator system with 1:3 internal resonance and parametric principal resonance[J]. Applied Mathematics and Mechanics (English Edition), 2015, 36(8): 985-1004.
[8]	Xuefen ZHAO, Xing LI, Shenghu DING. Two kinds of contact problems in three-dimensional icosahedral quasicrystals[J]. Applied Mathematics and Mechanics (English Edition), 2015, 36(12): 1569-1580.
[9]	Lei HOU, Yushu CHEN. Bifurcation analysis of aero-engine's rotor system under constant maneuver load[J]. Applied Mathematics and Mechanics (English Edition), 2015, 36(11): 1417-1426.
[10]	温登哲;陈予恕. Bifurcation analysis of fan casing under rotating air flow excitation[J]. Applied Mathematics and Mechanics (English Edition), 2014, 35(9): 1099-1114.
[11]	马锐陈予恕. Bifurcation of multi-freedom gear system with spalling defect[J]. Applied Mathematics and Mechanics (English Edition), 2013, 34(4): 475-488.
[12]	K. MALEKZADEH FARD M.M.MONFARED K. NOROUZIPOUR. Determination of stress intensity factors in half-plane containing several moving cracks[J]. Applied Mathematics and Mechanics (English Edition), 2013, 34(12): 1535-1542.
[13]	李忠刚;陈予恕. Research on 1:2 subharmonic resonance and bifurcation of nonlinear rotor-seal system[J]. Applied Mathematics and Mechanics (English Edition), 2012, 33(4): 499-510.
[14]	李军陈予恕. Transition sets of bifurcations of dynamical systems with two state variables with constraints[J]. Applied Mathematics and Mechanics (English Edition), 2012, 33(2): 139-154.
[15]	钟顺陈予恕. Bifurcation of elastic tank-liquid coupled sloshing system[J]. Applied Mathematics and Mechanics (English Edition), 2011, 32(9): 1169-1176.