I believe that the derivative of a Gaussian process (GP) is a another GP, and so I would like to know if there are closed form equations for the prediction equations of the derivative of a GP? In particular, I am using the squared exponential (also called the Gaussian) covariance kernel and want to know about making predictions about the derivative of the Gaussian process.
$\begingroup$
$\endgroup$
6
-
$\begingroup$ What do you mean by the derivative of the GP? do you randomly generate a curve from the BP, $x(t)$, and then take the derivative? $\endgroup$– PlacidiaNov 9, 2015 at 3:00
-
$\begingroup$ @Placidia, no I mean calculating $\frac{\partial x(t)}{\partial t}$, which I believe should be another Gaussian process $\endgroup$– user30490Nov 9, 2015 at 5:23
-
$\begingroup$ Good question. However I seem to recall that Brownian motion is both a GP and nowhere differentiable. So I'm not sure that there could be a generic expression. Of course x(t)-x(t-h) should be a Gaussian so it should be possible given the covariance function to think of probabilities about it for a given h. $\endgroup$– conjecturesNov 9, 2015 at 16:19
-
1$\begingroup$ @conjectures, that's why I specifically said I have a GP where the kernel function is the squared exponential (since I know that that one is infinitely differentiable) and was really only looking for the derivative case in my example. But good point none the less! $\endgroup$– user30490Nov 9, 2015 at 16:55
-
$\begingroup$ In case someone else looking for the derivative of Gaussian process, here you can find detail explanation mlg.eng.cam.ac.uk/mchutchon/DifferentiatingGPs.pdf $\endgroup$– GPrathapDec 28, 2021 at 7:37
|
Show 1 more comment
2 Answers
$\begingroup$
$\endgroup$
0
The short answer: Yes, if your Gaussian Process (GP) is differentiable, its derivative is again a GP. It can be handled like any other GP and you can calculate predictive distributions.
But since a GP $G$ and its derivative $G'$ are closely related you can infer properties of either one from the other.
- Existence of $G'$
A zero-mean GP with covariance function $K$ is differentiable (in mean square) if $K'(x_1, x_2)=\frac{\partial^2 K}{\partial x_1 \partial x_2}(x_1,x_2)$ exists. In that case the covariance function of $G'$ is equal to $K'$. If the process is not zero-mean, then the mean function needs to be differentiable as well. In that case the mean function of $G'$ is the derivative of the mean function of $G$.
(For more details check for example Appendix 10A of A. Papoulis "Probability, random variables and stochastic processes")
Since the Gaussian Exponential Kernel is differentiable of any order, this is no problem for you.
- Predictive distribution for $G'$
This is straightforward if you just want to condition on observations of $G'$: If you can calculate the respective derivatives you know mean and covariance function so that you can do inference with it in the same way as you would do it with any other GP.
But you can also derive a predictive distributions for $G'$ based on observations of $G$. You do this by calculating the posterior of $G$ given your observations in the standard way and then applying 1. to the covariance and mean function of the posterior process.
This works in the same manner the other way around, i.e. you condition on observations of $G'$ to infer a posterior of $G$. In that case the covariance function of $G$ is given by integrals of $K'$ and might be hard to calculate but the logic is really the same.
$\begingroup$
$\endgroup$
3
It is. See Rasmussen and Williams section 9.4. Also, some authors argue strongly against the square exponential kenrnel - it is too smooth.
-
1$\begingroup$ So is there a predictive distribution for the derivative? $\endgroup$– user30490Nov 9, 2015 at 5:24
-
$\begingroup$ Yes! Gradients of Gaussian processes are widely studied in wombling: en.wikipedia.org/wiki/Wombling $\endgroup$ 1 hour ago
-
$\begingroup$ You can check the developments in these papers: doi.org/10.1198/C16214503000000909 and in doi.org/10.1198/016214506000000041 $\endgroup$ 1 hour ago