I am performing Gaussian Process Regression (without noise) for response functions which are piecewise linear.

My question: Does there exist a covariance function, such that sample paths from a centred, stationary Gaussian Process with such a covariance are piecewise linear?

Currently I am using the D-dimensional "Hat"-Kernel $K(x,x')=\sigma^2 \prod_{d=1}^D\max(0,1 - \frac{|x_d - x_d'|}{l_d})$, which is one of the "Piecewise Polynomial Covariance Functions with Compact Support" found in the Rasmussen/Williams book. The mean function of a centred Gaussian Process with this covariance is piecewise linear.

Alas, the sample paths itself are incredibly wiggly. Hence inconsistent with the piecewise linear response functions I would like to model. This wiggliness can be easily seen from plotting samples or from the fact that such paths will have an infinite number of upcrossings (see the remark after equation (4.3) in chapter 4.1 of Rasmussen/Williams) in all neighbourhoods of a point of the domain. In light of this result and since piecewise linear functions are differentiable almost everywhere (wrt Lebesgue measure), the process needs to be differentiable in mean square without almost all paths being differentiable.


If the locations of the knots (breakpoints) would be constants, the joint distribution of $x,x+\delta$ would depend on whether there is a breakpoint in the interval, not only on $\delta$ as stationarity requires. Therefore, I assume the question intends the knots to be random, as well. In that case, obtaining piecewise linear sample paths with positive probability is not possible, as shown below.

I consider the one-dimensional case and denote the value of the Gaussian process at input $x_k$ by $f(x_k)$.

If the sample paths are piecewise linear with random breakpoint locations, we must have three distinct inputs $x_1,x_2,x_3$ so that the event 'The three points $(x_1,f(x_1)),(x_2,f(x_2)),(x_3,f(x_3))$ are on the same line' has a probability that is nonzero and below one. \begin{equation} \frac{f(x_2) - f(x_1)}{x_2 - x_1} = \frac{f(x_3) - f(x_2)}{x_3 - x_2} \end{equation} or equivalently \begin{equation} (x_2 - x_3)\,f(x_1) + (x_3 - x_1)\, f(x_2) + (x_1 - x_2)\,f(x_3) = 0. \end{equation} That is, the probability of the points being on the same line is $Pr(Z=0)$ where \begin{equation} Z := (x_2 - x_3)\,f(x_1) + (x_3 - x_1)\, f(x_2) + (x_1 - x_2)\,f(x_3). \end{equation} Note that $Z$ is a linear combination of $f(x_1),f(x_2),f(x_3)$. Since $f$ is a Gaussian process, the joint distribution of $(f(x_1),f(x_2),f(x_3))$ is multivariate Gaussian and therefore the distribution $Z$ is Gaussian. However, for a Gaussian $Z$, $Pr(Z=0)$ is either $0$ or $1$, a contradiction.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.