2
$\begingroup$

Is it possible to use a Gaussian Process to relate multiple independent input variables (X1, X2, X3) to an output variable (Y)?

More specifically, I would like to produce a regression graph like the example shown below where confidence interval reduces around clusters of data (i.e. variance is high at x = 1 where there is no data, but x = 0.3 the regression is tight due to the clustering of input variables) and Instead of having one input variable on the x-axis, where would be multiple inputs.

enter image description here

For example, is it possible to develop a regression relationship that relates the price of houses (HPrice = [125000, 63000, 500000]) to Floor Area (FArea = [856,497,1300]) and the Number of Bedrooms (BedR = [2,2,4])?

Ideally I would like to do this in R, and wonder if there are any recommendations/example available?

Thanks!

$\endgroup$
  • $\begingroup$ Could you please explain what you have in mind by "using a Gaussian process" and how that would differ from multiple regression? $\endgroup$ – whuber Feb 27 '14 at 18:58
  • 1
    $\begingroup$ Thanks @whuber - I've updated my post so that it is more specific. $\endgroup$ – hoof_hearted Feb 27 '14 at 19:42
4
$\begingroup$

Yes, it is entirely possible. Recall the Gaussian Process model is defined by a kernel function, $K(\bf x,\bf x')$, yours is a case where you need some function that exploits the vector $\bf x$ in an appropriate way.

The graphs in books tend to be univariate input (not always) just because it's straightforward to see what's going on. This is an example of one draw from a GP where the inputs are coordinates $\bf{x}$$=(x_1,x_2)$ on a $30 \times 30$ grid with a kernel I hoped would induce a spatial relationship.

enter image description here

I was reading Bishop 2006 at the time and found it helpful if you want more on the theory - which covers the regression also.

$\endgroup$
  • $\begingroup$ Could you provide the actual link to Bishop 2006? I'm unable to find this resource. Could you give some pointers on what adaptations are necessary for multi-input GP? $\endgroup$ – Yu Chen Aug 26 '17 at 23:21
  • $\begingroup$ The full citation for the book is: Bishop, Christopher M. Pattern recognition and machine learning. Springer, 2006. There is probably a pdf floating around somewhere. You might also want to look at the site: gaussianprocess.org/gpml which has more info online. $\endgroup$ – conjectures Aug 29 '17 at 10:59
  • $\begingroup$ Would you happen to be able to share a test example of code in Python that produces the plot above? $\endgroup$ – Mathews24 Nov 22 '18 at 6:02

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.