Scope: The potential of AI to advance fundamental science is gaining global attention, with scientific machine learning emerging as a prominent computational paradigm. Physics-informed neural networks (PINNs), in particular, enable function approximation consistent with physical laws by incorporating mathematically expressible laws of nature into their training loss function. This capability has propelled PINNs to the forefront of scientific machine learning. 

However, the physics loss in PINNs gives rise to unique optimization challenges that impede training speed, solution precision, and model generalizability. Extensive empirical studies have shown that standard iterative gradient-based algorithms for optimizing PINNs can lead to performance that’s orders of magnitude slower than conventional numerical schemes—so much so that a prominent article questioned the realistic advantages machine learning approaches could bring in scientific domains. These challenges primarily arise from the highly rugged landscape of the physics loss, the spectral bias of deep neural networks that slows convergence to high-frequency field components, the vanishing gradients that can deceptively satisfy governing differential equations, and the presence of spurious local optima traps corresponding to fixed points of nonlinear dynamical systems. 

In this tutorial, we demonstrate the immense potential of gradient-free algorithms, including randomized neural networks and evolutionary algorithms, as alternatives to traditional local gradient-based methods for overcoming PINN optimization challenges. Of significant interest are the methods of neuroevolution which offer robust strategies for optimizing the multi-modal loss landscapes of PINN models. Recent advances in transfer evolutionary optimization present an especially promising direction—where evolutionary strategies trained in one physical domain are adapted to new tasks, thereby improving convergence and generalization across varied physics regimes. We shall further discuss hybrid approaches that synergize gradient descent and neuroevolution for discovering bespoke architectures and balancing multiple competing terms in physics-informed loss formulations. Finally, by leveraging principles of the Baldwin effect, we position neuroevolution not only as an optimizer but also as a meta-learner—capable of evolving generalizable PINN frameworks across problem families—highlighting its potential to advance the next generation of scientific machine learning.

This tutorial builds upon our recent survey article on the Evolutionary Optimization of PINNs accepted for publication at the IEEE Computational Intelligence Magazine. Our discussions shall be accompanied by simple pedagogical examples as well as more complex studies in modelling particulate systems, solid mechanics, fluid mechanics, and heat transfer. 

Topics: 

The turorial will cover the following learning objectives:

• Acquire a basic understanding of scientific machine learning and its salient features

• Understand the formulation of a basic physics-informed neural network (PINN) and how it differs from standard data-driven deep learning and other scientific machine learning approaches including neural operator networks and neural ODEs.

• Understand the unique optimization challenges associated with gradient-based PINN training.

• Learn how gradient-free optimization, including randomized neural nets, evolutionary algorithms, and their hybrids can play a significant role in overcoming PINN optimization challenges.

• Gain familiarity with potential applications of Evo-PINNs to various scientific domains.

Important Dates:

Tentative Tutorial Date: 21st June 2026