Gaussian Process Regression with wide confidence interval

Question

I used Gaussian Process Regression to predict a time series, what I have is sensor's readings that come every hour ( I have data for about 3 years) I chose a periodic kernel function which looks like this

$$ K(x,x′)=\sigma_1^2 \exp(−2\sin^2(π|x−x′|/24)/\mathcal{l_1}^2)+\sigma_2^2 \exp(−2\sin^2(π|x−x′|/8)/\mathcal{l_2}^2) $$

I got the following predictions However, the confidence interval looks very weird, (same size along the predictions, and wide even around the data), I can't understand why I am getting this answer, can anyone explain to me what's wrong in my work!?

Briefly, this is what I do :

1. I get the data (sensor readings) and prepare training ($$ y(N*1) $$) and testing data ($$ y_{*}(M*1) $$) from the readings and preparing two corresponding series of numbers $$ x=[1,2,3,....N] $$ and $$ x_*=[1,2,3,....M] $$
2. Estimate the parameters using maximum likelihood
3. Calculate $$ K(x,x) +\sigma_{nois}, K(x_*,x)$$ and $$ K(x_*,x_*)$$
4. Calculate $$ L=K(x_*,x)*{K(x,x)}^{-1} $$
5. Calculate $$ Prediction=L*y $$ and $$ Covariance =k(x_*,x_*)-L*K(x,x_*)$$
6. Calculate the confidence interval like this:

$$ lower=Prediction+0.95*sqrt(Covariance) $$ $$ upper=Prediction-0.95*sqrt(Covariance) $$

1. Do I do anything wrong?! what is the problem?!
2. Is it right to consider the input (the time) in this way? and for the prediction mode, if I am going to predict the coming 24 hours should my input be $$ x_{pre}=[1,2,3,....,24] $$

Answer 1

The credible interval gets larger the further you extrapolate from the data. In this case, the data density is approximately the same throughout, so that broadening is not observed, and the credible interval is dominated by the estimate of the assumed additive noise contaminating the observations ($\sigma_\mathrm{nois}$). If you extended the plot, say from -100 to +100, I suspect you would see the broadening.

The width of the band look about right for a 95% credible interval, it has about as many data points lying outside it as would be reasonably expected.

If you expect the noise depends on the attribute/feature, you need a heteroscedastic Gaussian process model.

Answer 2

First, I'd recommend you use an existing GP package such as GPML or GPy/GPflow and not implement this yourself.

Assuming that the implementation is correct, it suggests that the most likely solution is that your data is pretty much pure noise - and that this is more likely than it coming from your kernel for any set of hyperparameters.

You may need to revise your selection of kernel function. Your data looks like it was generated from a non-stationary process - your kernel function is stationary.