AI in Dynamical Systems
This paper introduces a novel two-stream deep model based on graph convolutional network (GCN) architecture and feed-forward neural networks (FFNN) for learning the solution of nonlinear partial differential equations (PDEs). The model aims at incorporating both graph and grid input representations using two streams corresponding to GCN and FFNN models, respectively. Each stream layer receives and processes its input representation. As opposed to FFNN which receives a grid-like structure, the GCN stream layer operates on graph input data where the neighborhood information is incorporated through the adjacency matrix of the graph. In this way, the proposed GCN-FFNN model learns from two types of input representations, i.e. grid and graph data, obtained via the discretization of the PDE domain. The GCN-FFNN model is trained in two phases. In the first phase, the model parameters of each stream are trained separately. Both streams employ the same error function to adjust their parameters by enforcing the models to satisfy the given PDE as well as its initial and boundary conditions on grid or graph collocation (training) data. In the second phase, the learned parameters of two-stream layers are frozen and their learned representation solutions are fed to fully connected layers whose parameters are learned using the previously used error function. The learned GCN-FFNN model is tested on test data located both inside and outside the PDE domain. The obtained numerical results demonstrate the appli- cability and efficiency of the proposed GCN-FFNN model over individual GCN and FFNN models on 1D- Burgers, 1D-Schrödinger, 2D-Burgers, and 2D-Schrödinger equations.
Learning solutions to partial differential equations using LS-SVM [Neurocomputing 2015][Github][PDF]
This paper proposes an approach based on Least Squares Support Vector Machines (LS-SVMs) for solving second order partial differential equations(PDEs) with variable coefficients. Contrary to most existing techniques, the proposed method provides a closed form approximate solution. The optimal representation of the solution is obtained in the primal–dual setting. The model is built by incorporating the initial/boundary conditions as constraints of an optimization problem. The developed method is well suited for problems involving singular, variable and constant coefficients as well as problems with irregular geometrical domains. Numerical results for linear and nonlinear PDEs demonstrate the efficiency of the proposed method over existing methods.
In this paper, a new approach based on least squares support vector machines (LS-SVMs) is proposed for solving linear and nonlinear ordinary differential equations (ODEs). The approximate solution is presented in closed form by means of LS-SVMs, whose parameters are adjusted to minimize an appropriate error function. For the linear and nonlinear cases, these parameters are obtained by solving a system of linear and nonlinear equations, respectively. The method is well suited to solving mildly stiff, nonstiff, and singular ODEs with initial and boundary conditions. Numerical results demonstrate the efficiency of the proposed method over existing methods.[PDF1, PDF2, PDF3].
LS-SVM approximate solution to linear time varying descriptor systems [Automatica 2012][Matlab Code][PDF]
This paper discusses a numerical method based on Least Squares Support Vector Machines (LS-SVMs) for solving linear time varying initial and boundary value problems in Differential Algebraic Equations (DAEs). The method generates a closed form (model-based) approximate solution. The results of numerical experiments on different systems with index from 0 t0 3, are presented and compared with analytic solutions to confirm the validity and applicability of the proposed method.
This paper introduces an estimation method based on Least Squares Support Vector Machines (LS-SVMs) for approximating time-varying as well as constant parameters in deterministic parameter-affine delay differential equations (DDEs). The proposed method reduces the parameter estimation problem to an algebraic optimization problem. Thus, as opposed to conventional approaches, it avoids iterative simulation of the given dynam- ical system and therefore a significant speedup can be achieved in the parameter estima- tion procedure. The solution obtained by the proposed approach can be further utilized for initialization of the conventional nonconvex optimization methods for parameter estima- tion of DDEs. Approximate LS-SVM based models for the state and its derivative are first estimated from the observed data. These estimates are then used for estimation of the unknown parameters of the model. Numerical results are presented and discussed for demonstrating the applicability of the proposed method.
Parameter Estimation for Time Varying Dynamical Systems using Least Squares Support Vector Machines [16th IFAC Symposium on System Identification, 2012][PDF][Github]
This paper develops a new approach based on Least Squares Support Vector Machines (LS-SVMs) for parameter estimation of time invariant as well as time varying dynamical SISO systems. Closed-form approximate models for the state and its derivative are first derived from the observed data by means of LS-SVMs. The time-derivative information is then substituted into the system of ODEs, converting the parameter estimation problem into an algebraic optimization problem. In the case of time invariant systems one can use least-squares to solve the obtained system of algebraic equations. The estimation of time-varying coefficients in SISO models, is obtained by assuming an LS-SVM model for it.
In this study the estimation of parameters in dynamical systems governed by parameter-affine ordinary differential equations is explored. The described method by Mehrkanoon et al.~ in [1] is utilized as an initialization of the nonlinear optimization problem for parameter estimation. In contrast to existing convex initialization approaches [2] that use a first order Euler discretization, we do not require any integration method to simulate the dynamical system. Furthermore, a denoising scheme using LSSVM is proposed to first filter the measured data then proceed with the filtered signals for parameter estimation problem. Experimental results demonstrate the efficiency of the proposed method, compared to alternative approaches on different examples from the literature.
Although time delay is an important element in both system identification and control performance assessment, its computation remains elusive. This paper proposes the application of a least squares support vector machines driven approach to the problem of determining constant time delay for a chemical process. The approach consists of two steps, where in the first step the state of the system and its derivative are approximated based on the LS-SVM model. The second stage consists of modeling the delay term and estimating the unknown model parameters as well as the time delay of the system. Therefore the proposed approach avoids integrating the given differential equation that can be computationally expensive. This time delay estimation method is applied to both simulation and experimental data obtained from a continuous, stirred, heated tank. The results show that the proposed method can provide accurate estimates even if significant noise or unmeasured additive disturbances are present.
A software tool SYM-LS-SVM-SOLVER written in Maple is developed to derive the dual system and the dual model representation of LS-SVM based models, symbolically. SYM-LS-SVM-SOLVER constructs the Lagrangian from the given objective function and list of constraints. Afterwards it obtains the KKT (Karush-Kuhn-Tucker) optimality conditions and finally formulates a linear system in terms of the dual variables. The snapshot of the GUI of the developed solver is shown in the following Figure.
