A feature of the Bayesian approach to Gaussian processes is the fact that asymptotic behaviour far away from data is controlled by the prior mean function. In most cases covered in the literature the prior mean is either a function independent of the data (such as a constant, often zero) or a function in a low dimensional function space, such as a polynomial, whose posterior parameters are specified by data.
Both of these approaches (constant or few parameters) do not work in my application. Prior knowledge tells me the mean function is asymptotically linear, i.e. its second derivative vanishes, but the gradient will depend on the data. Hence constant priors will not work: The gradient is not necessarily zero. Parametrising the mean is not feasible, since my problem is high dimensional (multivariate time series data, dimension on the order of hundreds) and a reasonable parametrisation would require hundreds of parameters, leading to overfitting.
Being twice differentiable almost everywhere is not a problem in my application, hence it would be great if, instead of specifying mean zero or $f=0$, I could specify $f''=0$ as a prior mean. Theoretically this should work since $f''$ is also a Gaussian process and I could condition $f''$ on observations of $f$. Practically. I have a problem, since the correlations of $f$ and $f''$ involve high dimensional double integrals of the kernel, which seem infeasible to calculate.
So my question is: What is the best way to approach Gaussian regression, in case your prior belief about the process is that it is asymptotically linear or $f''=0$?