I built a Gaussian Process model to perform regression. A set of "known" points was used (each point has a "true function" value). After that, the model is able to predict values in other "unknown" points.

If the predicted value for some point P of these "unknown" points is taken as the "true value", and P is added to the set of "known" points, would a new GP model based in this augmented set of points present any difference in its predictions?

In my tests, the variance of the model changes, but the predictions remain the same, no matter how many points are added (in the same way P was added).


1 Answer 1


You can check your suggestion by the direction application of the matrix inverse formula.

The new covariance matrix for the sample has the form: $$ \tilde{K} = \begin{pmatrix} K & \mathbf{k} \\ \mathbf{k}^T & c \\ \end{pmatrix} $$

Now we apply the block matrix inverse formula (see e.g. http://www.cs.nthu.edu.tw/~jang/book/addenda/matinv/matinv/): $$ \tilde{K}^{-1} = \begin{pmatrix} K^{-1} + \frac{1}{c} K^{-1} \mathbf{k} \mathbf{k}^T K^{-1} & -\frac{1}{c} K^{-1} \mathbf{k} \\ -\frac{1}{c} K^{-1} \mathbf{k} & \frac{1}{c} \\ \end{pmatrix} $$

Then after using the common formula for the posterior mean at a new point all terms except $\mathbf{k}_{new} K^{-1} \mathbf{y}$ cancel, and we get the same mean.

If we'll continue to add new points with the suggested mean values then mean will not change. To prove apply the formulas above and the induction.

P.S. May be a less technical proof exists.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.