I realize this is a very old question, but since this is one of the google results on this topic, I'd like to also point out that there is some literature (from the 90s, but resuming in 2018) on learning not the solution to a given PDE or ODE, but learning the PDE or ODE itself. See work from 1989 through 1998 by Kevrekidis, Rico-Martinez, Farber, and others.
For example, see , which compares several methods for ODEs, but see in particular section III3 and Figure 9.
For PDEs, see , in particular Figure 3, which you may recognize as a PDE version of the four-state Runge-Kutta Figure 9 of .
The approach of this family of papers is to establish the learning problem as predicting the short-time flow of the ODE, and finding the ODE that does that. But the ODE is not part of the input--it is the thing which is learned. The data which is fed to the network is only the state before and after the short flow time. The fact that the composite neural network can predict the state at time T is not the point; the useful artifact of the training is the sub-network which evaluates the ODE or PDE.
Likely this is not what you really wanted--several other works (and more, and more) learn the solution, not the ODE. An advantage of these methods over standard numeric timesteppers was pointed out to be their parallelism-in-time.
If you're in fact interested in analytical solutions, there is some more recent work on training a "tree-LSTM" to produce the correct analytical solution when presented with an higher-order ODE, though I'm not very familiar with this approach, the form that "presenting the ODE" takes, and how widely applicable this method is.
 Rico-Martínez, Ramiro, Katharina Krischer, Ioannis G Kevrekidis, M. C. Kube, and J. L. Hudson. “Discrete- vs Continuous-Time Nonlinear Signal Processing of Cu Electrodissolution Data.” Chemical Engineering Communications 118, no. 1 (1992): 25–48. https://doi.org/10.1080/00986449208936084.
 González-García, R., Ramiro Rico-Martínez, and Ioannis G Kevrekidis. “Identification of Distributed Parameter Systems: A Neural Net Based Approach.” Computers & Chemical Engineering 22, no. 98 (March 1998): S965–68. https://doi.org/10.1016/S0098-1354(98)00191-4.