I am reading this cool paper about Neural SDEs as GANs. I've gotten through all of it and I understand fairly well. I've taken a couple classes on SDEs so I'm comfortable with the math. What I don't have any experience with are SDE solvers. Part of the training algorithm is solving an SDE with an SDE solver. Then it is described in one or two sentences that backprop is performed "through the internal operations of the numerical SDE solver." I'm trying to understand what this means?
The point is that the drift and diffusion terms of the SDE are neural net functions of some parameters, and these parameters need to be optimized. During one iteration of training the parameters are fixed and we need to find a solution of an SDE based on these parametrized functions. The solution is then used to compute a sample path (output) of the model. I'm just not quite understanding how we can backprop the gradients through this solver? I guess I need to read more about SDE solvers in general.