1
$\begingroup$

When performing Gaussian process regression, the variance at a prediction point is given by $var[f_*] = k(x_*,x_*) - k_*^T(k+\sigma_n^2I)^{-1}k_*$ (Equation 2.26 from GPML)

The variance is not dependent on the observed targets, but only on the inputs. Does that mean that the targets need to be scaled? If so, what's the appropriate scaling? Otherwise, it seems wrong the relative variance changes depending on what units or scale are used for the targets.

$\endgroup$

1 Answer 1

2
$\begingroup$

The contradiction you arrived at makes sense. This sentence should be read with caution:

The variance is not dependent on the observed targets

The hyperparameters associated with the covariance function learnt during the training would depend on the observed targets. That way the target values affect the prediction variance indirectly. You can verify that by running a small demo in gpml. Think about it ... The covariance function should consider most training inputs with similar target values similar and training inputs with dissimilar target values dissimilar. This is achieved by learning appropriate hyperparameters associated with the covariance function.

Once the hyperparameters are learnt, the target values play no further role as equation 2.26 indicates.

Suppose another scenario where the hyperparameters of the covariance function are known a priori therefore they are not to be learnt during the training. In such a scenario target values play no role at all.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.