Using Neural Networks with Numerical Methods to Solve Complex Differential Equations

This research investigates the use of neural networks in combination with the numerical methods and their application to the solution of the complex differential equations. Many of the scientific and engineering phenomena that are commonly modeled with differential equations can't be solved analytically because of complex boundary conditions, high dimension, stiffness, or nonlinearity. Thus, this study aims to fairly compare the numerical methods with the physics-informed neural network and hybridized neural–numerical methods, in terms of their accuracy, convergence, stability and computational efficiency. The methodology used is analytical–computational using selected test problems with known exact solutions namely, an ordinary differential equation (ODE), a boundary value problem (BVP) and a partial differential equation (PDE). It shows that neural networks are efficient at giving flexible approximation in solving the problem while the classical numerical methods are efficient in solving structured problems. The hybrid neural-numerical method gives the best overall performance in which both the stable numerical method and learning ability of neural networks are combined. In this study the conclusion is drawn that a neural network should not be used to replace traditional methods, but should be implemented alongside, to complement.

Neural Networks; Numerical Methods; Differential Equations; Physics-Informed Neural Networks; Hybrid Neural–Numerical Method; Computational Accuracy; Convergence; Stability.

1. R. L. Burden, J. D. Faires, and A. M. Burden, Numerical Analysis, 10th ed. Boston, MA, USA: Cengage Learning, 2016. [Google Scholar] [Crossref]

2. R. J. LeVeque, Finite Difference Methods for Ordinary and Partial Differential Equations: Steady-State and Time-Dependent Problems. Philadelphia, PA, USA: SIAM, 2007. [Google Scholar] [Crossref]

3. C. Canuto, M. Y. Hussaini, A. Quarteroni, and T. A. Zang, Spectral Methods: Fundamentals in Single Domains. Berlin, Germany: Springer, 2006. [Google Scholar] [Crossref]

4. G. Cybenko, “Approximation by superpositions of a sigmoidal function,” Mathematics of Control, Signals and Systems, vol. 2, pp. 303–314, 1989. [Google Scholar] [Crossref]

5. K. Hornik, M. Stinchcombe, and H. White, “Multilayer feedforward networks are universal approximators,” Neural Networks, vol. 2, no. 5, pp. 359–366, 1989. [Google Scholar] [Crossref]

6. I. E. Lagaris, A. Likas, and D. I. Fotiadis, “Artificial neural networks for solving ordinary and partial differential equations,” IEEE Transactions on Neural Networks, vol. 9, no. 5, pp. 987–1000, 1998. [Google Scholar] [Crossref]

7. M. Raissi, P. Perdikaris, and G. E. Karniadakis, “Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations,” Journal of Computational Physics, vol. 378, pp. 686–707, 2019. [Google Scholar] [Crossref]

8. J. Sirignano and K. Spiliopoulos, “DGM: A deep learning algorithm for solving partial differential equations,” Journal of Computational Physics, vol. 375, pp. 1339–1364, 2018. [Google Scholar] [Crossref]

9. M. W. M. G. Dissanayake and N. Phan-Thien, “Neural-network-based approximations for solving partial differential equations,” Communications in Numerical Methods in Engineering, vol. 10, no. 3, pp. 195–201, 1994. [Google Scholar] [Crossref]

10. W. E and B. Yu, “The Deep Ritz Method: A deep learning-based numerical algorithm for solving variational problems,” Communications in Mathematics and Statistics, vol. 6, no. 1, pp. 1–12, 2018. [Google Scholar] [Crossref]

11. J. Han, A. Jentzen, and W. E, “Solving high-dimensional partial differential equations using deep learning,” Proceedings of the National Academy of Sciences, vol. 115, no. 34, pp. 8505–8510, 2018. [Google Scholar] [Crossref]

12. L. Lu, X. Meng, Z. Mao, and G. E. Karniadakis, “DeepXDE: A deep learning library for solving differential equations,” SIAM Review, vol. 63, no. 1, pp. 208–228, 2021. [Google Scholar] [Crossref]

13. Z. Li, N. Kovachki, K. Azizzadenesheli, B. Liu, K. Bhattacharya, A. Stuart, and A. Anandkumar, “Fourier Neural Operator for parametric partial differential equations,” in Proc. International Conference on Learning Representations (ICLR), 2021. [Google Scholar] [Crossref]

14. Z. Uddin, S. Ganga, R. Asthana, and W. Ibrahim, “Wavelets based physics informed neural networks to solve non-linear differential equations,” Scientific Reports, vol. 13, Art. no. 2882, 2023. [Google Scholar] [Crossref]

15. B. Moseley, A. Markham, and T. Nissen-Meyer, “Finite basis physics-informed neural networks (FBPINNs): A scalable domain decomposition approach for solving differential equations,” Advances in Computational Mathematics, vol. 49, Art. no. 62, 2023. [Google Scholar] [Crossref]

16. Q. Fang, X. Mou, and S. Li, “A physics-informed neural network based on mixed data sampling for solving modified diffusion equations,” Scientific Reports, vol. 13, Art. no. 2491, 2023. [Google Scholar] [Crossref]

17. A. M. Hafiz, I. Faiq, and M. Hassaballah, “Solving partial differential equations using large-data models: A literature review,” Artificial Intelligence Review, vol. 57, Art. no. 152, 2024. [Google Scholar] [Crossref]

18. Q. Cao, S. Goswami, and G. E. Karniadakis, “Laplace neural operator for solving differential equations,” Nature Machine Intelligence, vol. 6, pp. 631–640, 2024. [Google Scholar] [Crossref]

19. M. Hintermüller and D. Korolev, “A hybrid physics-informed neural network based multiscale solver as a partial differential equation constrained optimization problem,” arXiv preprint arXiv:2309.04439, 2023. [Google Scholar] [Crossref]

20. X. Feng, H. Shangguan, T. Tang, X. Wan, and T. Zhou, “A hybrid FEM-PINN method for time-dependent partial differential equations,” arXiv preprint arXiv:2409.02810, 2024. [Google Scholar] [Crossref]

21. W. Zhang, W. Suo, J. Song, and W. Cao, “Physics informed neural networks as intelligent computing technique for solving partial differential equations: Limitation and future prospects,” arXiv preprint arXiv:2411.18240, 2024. [Google Scholar] [Crossref]

• Technology Integration among Micro, Small, and Medium Enterprises’ (MSMEs) Operations in Davao Oriental, Philippines