3
$\begingroup$

Most internet resources--- papers, slides, etc--- on GP I've found take the mean function to be zero.
Chapter 15 of Murphy's book (Machine Learning: A Probabilistic Perspective) says this is because 'the GP is flexible enough to model the mean arbitrarily well, as we will see below'.
I didn't see that he ended up explaining this comment.

So my question is:
Why do you not lose generality when you make a zero mean assumption?

$\endgroup$

3 Answers 3

3
$\begingroup$

Change of coordinates and stationarity.

Lets say that $ f(x)$ is a non-standard Gaussian distribution with mean $ \mu$ then there by changing the domain to $ y = x - \mu$ the function $ f(y)$ has zero mean.

If the distribution is stationary then this transformation allows validates the zero mean assumption. If you work in terms of y and g, then you are not necessarily incorrect.

$\endgroup$
1
$\begingroup$

I had the same question and now figure it out.

The zero-mean assumption is on the prior distribution p(f,f*), the posterior p(f,f*|X) will have an updated non-zero mean according to the observation X.

Murphy's book also introduced some mean functions, but it is just for an easier interpretation of the modelling. It doesn't contribute to the accuracy of the model.

$\endgroup$
1
$\begingroup$

You're not losing generality because the mean function commutes with the training. That is, if you subtract your mean up-front and and eventually add it to your predictions, you will get the same predictive distribution as if you had started with a prior distribution with non-zero mean and computed the conditional distributions from there.

As far as Murphy's comment about flexibility goes, keep in mind that the posterior mean of a Gaussian process with kernel $K$ is always an element of the RKHS associated with that kernel, i.e. for an SE/RBF kernel, the posterior mean will always be $C^\infty$ smooth; for an exponential kernel, it will always be a $H^1$ Sobolev function (will be $C^1$ smooth, but with kinks at the training data points), etc. (cf. Rasmussen/Williams for the relation between Gaussian processes and RKHSs). RKHSs are vector spaces, so they are invariant with respect to addition/subtraction of a mean function as long as that mean function is an element of that RKHS.

However, it does make a difference if you know a prior mean and decide to ignore it. You will get inaccurate predictions then.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.