2
$\begingroup$

I am trying to solve some Ordinary Differential Equations using Neural Networks. I have read quite a few papers and even some dissertations on the same. However I am a little unclear on how the neural network itself is trained - what are the inputs, what are the target outputs, do we need to write the backpropogation algorithm or can we use some pre-existing libraries for it. Basically can someone clarify with an example the role of the neural network in solving ordinary differential equations?

Thanks in advance!

$\endgroup$
2
  • $\begingroup$ Can the ODE be solved by other means? If not what is an example of an ODE that was solved with a neural network. $\endgroup$ – Michael R. Chernick May 27 '17 at 2:26
  • $\begingroup$ Woah, this is what got David Duvenaud the best paper award at neurips 2018. Here is a link to the paper -arxiv.org/abs/1806.07366. It also links to the github repo where the code to a differentiable ODESOlver is available! $\endgroup$ – Aastha Dua Dec 21 '19 at 6:28
2
$\begingroup$

I have seen application of machine learning algorithms, like the Stochastic approximation Expectation Maximization (SAEM) that solve ODES that cannot be solved analytically. If you know the equations defining your dynamic system but not the values of the constants in them, you could use such algorithms to do Parameter Inference.

I am guessing you could also apply Nueral Networks for it. Could you offer the papers and dissertations you have read? I can try to explain them.

$\endgroup$
2
$\begingroup$

I realize this is a very old question, but since this is one of the google results on this topic, I'd like to also point out that there is some literature (from the 90s, but resuming in 2018) on learning not the solution to a given PDE or ODE, but learning the PDE or ODE itself. See work from 1989 through 1998 by Kevrekidis, Rico-Martinez, Farber, and others.

For example, see [1], which compares several methods for ODEs, but see in particular section III3 and Figure 9.

For PDEs, see [2], in particular Figure 3, which you may recognize as a PDE version of the four-state Runge-Kutta Figure 9 of [1].

The approach of this family of papers is to establish the learning problem as predicting the short-time flow of the ODE, and finding the ODE that does that. But the ODE is not part of the input--it is the thing which is learned. The data which is fed to the network is only the state before and after the short flow time. The fact that the composite neural network can predict the state at time T is not the point; the useful artifact of the training is the sub-network which evaluates the ODE or PDE.

Likely this is not what you really wanted--several other works (and more, and more) learn the solution, not the ODE. An advantage of these methods over standard numeric timesteppers was pointed out to be their parallelism-in-time.

If you're in fact interested in analytical solutions, there is some more recent work on training a "tree-LSTM" to produce the correct analytical solution when presented with an higher-order ODE, though I'm not very familiar with this approach, the form that "presenting the ODE" takes, and how widely applicable this method is.

[1] Rico-Martínez, Ramiro, Katharina Krischer, Ioannis G Kevrekidis, M. C. Kube, and J. L. Hudson. “Discrete- vs Continuous-Time Nonlinear Signal Processing of Cu Electrodissolution Data.” Chemical Engineering Communications 118, no. 1 (1992): 25–48. https://doi.org/10.1080/00986449208936084.

[2] González-García, R., Ramiro Rico-Martínez, and Ioannis G Kevrekidis. “Identification of Distributed Parameter Systems: A Neural Net Based Approach.” Computers & Chemical Engineering 22, no. 98 (March 1998): S965–68. https://doi.org/10.1016/S0098-1354(98)00191-4.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.