0
$\begingroup$

I know Gaussian Progress Regression is completed determined by its mean and covariance functions. For given training and test data, I can compute the covariance matrix correctly like a machine, and I can also describe the main procedure of GPR (compute the joint probability over training data and estimation point, cut a piece through the marginal likelihood....etc), but could anyone can describe what is Gaussion Progress for Regression using your OWN words?

$\endgroup$
0
$\begingroup$

I'm far from being an expert, but here I go. The posterior mean function in a GP regression is a linear smoother, and it can be seen as a generalized nearest-neighbors prediction method: for any point $x^*$ in regressor space, a mean predicted output $\hat y^*$ is computed from the training data $(\mathbf{X}, \mathbf{Y})= (x_i, y_i)_{i \in [1, n]}$ as a weighted average:

$\hat y^*(x^*, \mathbf{X}, \mathbf{Y}) = \sum_{i=1}^n w_i(x^*, \mathbf{X}) \cdot y_i$

where the weights $w_i$ will typically be high for training observations "close" to point x$^*$ (in regressor space, ie. using only $\mathbf{X})$, and low for "far away" observations.

Changing the covariance function (kernel) and hyperparameters changes the definition of how close two observations $(x_1, x_2)$ are, and this will change the weights $w_i$. But in the end the mean prediction $\hat y^*$ is always just an average of the observed $y_i$ associated with observed $x_i$ that are close to $x^*$.

Additionally to this mean prediction $\hat y^*$, GP regression gives you the (Gaussian) distribution of $y$ around this mean, which will be different at each query point $x^*$ (in contrast with ordinary linear regression for instance, where only the predicted mean of $y$ changes with $x$ but where its variance is the same at all points).

A fitted GP even gives you the full joint distribution of any vector $(y^*_1, y^*_2, ... y^*_t)$ given the query inputs $(x^*_1, x^*_2, ... x^*_t)$, which is really a lot of information (including estimated partial derivatives $\widehat{\frac{\partial y}{\partial x}}$), but for regression you're probably mostly interested in the predicted mean and variance.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.