1
$\begingroup$

The Gaussisna process (GP) is one of the most famous non-parametric regression techniques. The gaussians process is defined as a collection of random variablers with a joint multivariate gaussian distribution.

I have read that the gaussian process is non-monotone. Is that always true, i.e. is the GP non-monotone with probability 1 ?

Is there a way to show that the Gaussian process can be monotone is some cases i.e (for any $x\leq y$ we have that $f(x)\leq f(y)$. Or can this be disproven ?

$\endgroup$
6
  • $\begingroup$ For the benefit of the readers, you should say exactly what you mean by non-monotone, in the context of a Gaussian Process. $\endgroup$ Sep 18, 2016 at 17:12
  • $\begingroup$ @MarkL.Stone I have added some decription to adress your comment $\endgroup$
    – Wis
    Sep 18, 2016 at 19:04
  • $\begingroup$ Are x and y vectors of dimension $ > 1$ (in general)? If so, what does $x \le y$ mean? $\endgroup$ Sep 18, 2016 at 19:13
  • $\begingroup$ @MarkL.Stone. They are just scalar values, I did not add a bold font for that reason. Thanks $\endgroup$
    – Wis
    Sep 18, 2016 at 19:33
  • $\begingroup$ The mean function of a Gaussian process (defined on $\mathbb{R}$) can be monotone: Just add a monotone function to a zero mean process. But since $f(x)$ and $f(y)$ are bivariate normal, the probability $P(f(x)>f(y))$ will be always larger than zero, no matter what covariance function you choose (except for degenerate cases such as correlation constant -1 or similar) $\endgroup$
    – g g
    Sep 18, 2016 at 22:19

1 Answer 1

1
$\begingroup$

Yes, a Gaussian Process (GP) is strictly non-monotonic. This is easy to show for the case that the kernel is non degenerate (ie full rank) as the posterior of any unobserved point will have a posterior distribution which is normally distributed between hence spans $[-\infty, \infty]$. As such the probability of $y < \hat{y}$ for any given $\hat{y}$ is strictly greater than zero.

However, there are hacks. There has been work done on modelling the derivative of a function space with a GP and given an appropriate mean function claims can be made about draws of the function space being approximately monotonic. You can also probably model some function of the derivative space such as the log or square root of the derivative, but inference will likely become more complex.

Potentially useful links:

Gaussian Processes with Monotonicity Constraints

Bayesian Monotone Regression using Gaussian Process Projection

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.