Symbolic Machine Learning
Overview
Learning high-dimensional functions (e.g., solving high-dimensional partial differential equations (PDEs) and discovering governing PDEs) is fundamental in scientific fields such as diffusion, fluid dynamics, and quantum mechanics, and optimal control, etc. Developing efficient and accurate solvers for this task remains an important and challenging topic. Traditional solvers (e.g., finite element method (FEM) and finite difference) are usually limited to low-dimensional domains since the computational cost increases exponentially in the dimension as the curse of dimensionality. Neural networks (NNs) as mesh-free parameterization are widely employed in solving regression problems and high-dimensional PDEs. Yet the highly non-convex optimization objective function in NN optimization makes it difficult to achieve high accuracy. The errors of NN-based solvers would still grow with the dimension. Besides, NN parametrization may still require large memory and high computation cost for high-dimensional problems. Finally, numerical solutions provided by traditional solvers and NN-based solvers are not interpretable, e.g., the dependence of the solution on variables cannot be readily seen from numerical solutions. The key to tackle these issues is to develop symbolic learning to discover the low-complexity structures of a high-dimensional problem. Low-complexity structures are applied to transform a high-dimensional task into a low-dimensional learning problem.
High-Dimensional PDEs
We propose the finite expression method (FEX) [pdf], a methodology that aims to find a solution in the function space of mathematical expressions with finitely many operators. Compared with existing methods, our FEX enjoys the following advantages (summarized in the figure below): (1) The expression may reproduce the true solution and achieve a solution with high and even machine accuracy. (2) The expression requires a low cost on memory (a line of string) for solution storage and computation for solution evaluation. (3) The expression has good interpretability with an explicit and legible form. Besides, from the approximation perspective, the expression in FEX is capable of avoiding the curse of dimensionality in theory with convincing numerical performance for high-dimensional Schrödinger equations [pdf] and committer functions [pdf].
Learning Operators & Governing Equations
We proposed a novel deep symbolic method to find the governing equations [pdf] in the function space of mathematical expressions generated by binary expression trees with a fixed number of operators, which is called the finite expression method (FEX). Compared to other symbolic approaches, our FEX enjoys the following advantages (summarized in the figure below): (1) Binary expression trees in FEX can generate a large class of mathematical expressions from a small number of operator list, avoiding the requirement of a large dictionary of symbols or a large symbolic neural network. (2) FEX can effectively discover nonlinear and non-polynomial equations of mathematical operators while other symbolic approaches fail. (3) The problem of searching for an appropriate mathematical expression is reformulated into a reinforcement learning problem, where effective optimization algorithms are applicable with high efficiency in memory cost. Therefore, FEX has the capacity to solve high-dimensional and high-order problems.
