0
$\begingroup$

I have a question about practiacal implementation and interpretation of the Gaussian process regression model given by Rasmussen & Williams (http://www.gaussianprocess.org/gpml).

The regression problem is defined as follows:

Let $x_i∈R^{5}$ be an input vector and $y_i∈R$ be its corresponding target. The set of $M$ inputs are arranged into a matrix $X=[x_1,…,x_M]^⊤$ and their corresponding targets are stored in a matrix $Y=[y_1,…,y_M]^⊤$

I wish to train a GPR model $G={X,Y,θ}$ using the squared exponential function:

$k(x_i,x_j)=α^2exp(\frac{−1}{2β^2(xi−xj)^2})+γ^2δ_{ij}$,

where $δ_{ij}$ equals 1 if $i=j$ and 0 otherwise. The hyperparameters are $θ=(α,β,γ)$ with γγ being the assumed noise level in the training data and $β$ is the length-scale.

To train the model, I need to minimise the negative log marginal likelihood with respect to the hyperparameters:

$−logp(Y∣X,θ)=\frac{1}{2}tr(Y^⊤K^{−1}Y)+\frac{1}{2}log∣K∣+c$, where c is a constant and the matrix K is a function of the hyperparameters

covfunc = @covSEiso;
likfunc = @likGauss;
sn = 0.1;
hyp.lik = log(sn);
hyp2.cov = [0;0];
hyp2.lik = log(0.1);
hyp2 = minimize(hyp2, @gp, -100, @infExact, [], covfunc, likfunc, X1, Y1(:,n));
exp(hyp2.lik)
nlml2 = gp(hyp2, @infExact, [], covfunc, likfunc, X1, Y1(:, n));
[m s2] = gp(hyp2, @infExact, [], covfunc, likfunc, X1, Y1(:, n), XT);
YT(:, n) = m;

X1,Y1 are the training inputs/targets

XT are the test inputs, YT are the target predictions.

My question is about the use of [m,s2] as predictions for Y (the second to last line of code). If the original sample Y are not normally distributed is the predictive distribution given by the values in [m,s2] stil valid? For example is it valid to say that for a test value $x_{*i}$ the distribution of the prediction for the target $y_{*i}$ is $N(m,s2)$, or am I misinterpreting the implementation of this model?

$\endgroup$
  • $\begingroup$ In Gaussian process regression (GPR) we assume that our function is a realization of Gaussian process. Almost any function is close to some Gaussian process realization. In particular, GPR can fit the training data perfectly i.e. pass through the all sample points. I have encountered no discussion regarding testing of Gaussian nature of data while using GPR. $\endgroup$ – Alexey Zaytsev Sep 29 '16 at 11:43
  • $\begingroup$ The assumption in GPR is that the noise is Gaussian. The sample Y will not typically be, but the residuals after you've applied your model should be if the model is well-specified. This is illustrated in the figure below, somewhat. There are cases for which the noise is truly non-gaussian so that even with a well-specified model the residuals will be non gaussian or heteroskedastic. In these cases, something like Warped Gaussian Processes by Snelson/Ghahramani can be used. There is much literature on the topic, they provide a few good references towards the end of the paper. $\endgroup$ – David Kozak Feb 4 at 18:17
0
$\begingroup$

For a gaussian process, the evaluation of the gaussian processes of an input value $(x+1)$ is the output $y(x+1) $ obtained by

$$\mathord{\buildrel{\lower3pt\hbox{$\scriptscriptstyle\frown$}} \over y} = {K^y}(\mathord{\buildrel{\lower3pt\hbox{$\scriptscriptstyle\frown$}} \over x} ,x)*{\left( {{K^Y}} \right)^{ - 1}}*y $$

in your code, this is expressed by $m$ ,the mean value is the value of higher probability. For each value that you want to evaluate, you need to calculate $${K^y}(\mathord{\buildrel{\lower3pt\hbox{$\scriptscriptstyle\frown$}} \over x} ,x) $$ and then you apply the equation for the mean value to obtain your output value.

You can see a gaussian process as a sum of gaussian distributions over a specific path. Gaussian Processes Plot

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.