3
$\begingroup$

Is it possible to use a Gaussian Process to relate multiple independent input variables (X1, X2, X3) to an output variable (Y)?

More specifically, I would like to produce a regression graph like the example shown below where confidence interval reduces around clusters of data (i.e. variance is high at x = 1 where there is no data, but x = 0.3 the regression is tight due to the clustering of input variables) and Instead of having one input variable on the x-axis, where would be multiple inputs.

enter image description here

For example, is it possible to develop a regression relationship that relates the price of houses (HPrice = [125000, 63000, 500000]) to Floor Area (FArea = [856,497,1300]) and the Number of Bedrooms (BedR = [2,2,4])?

Ideally I would like to do this in R, and wonder if there are any recommendations/example available?

Thanks!

$\endgroup$
3
  • $\begingroup$ Could you please explain what you have in mind by "using a Gaussian process" and how that would differ from multiple regression? $\endgroup$
    – whuber
    Commented Feb 27, 2014 at 18:58
  • 1
    $\begingroup$ Thanks @whuber - I've updated my post so that it is more specific. $\endgroup$ Commented Feb 27, 2014 at 19:42
  • $\begingroup$ I have the same question, all the examples are just show the same stupid example, which is y=sin(x) + N() $\endgroup$
    – Laha Ale
    Commented Aug 4, 2021 at 4:40

2 Answers 2

6
$\begingroup$

Yes, it is entirely possible. Recall the Gaussian Process model is defined by a kernel function, $K(\bf x,\bf x')$, yours is a case where you need some function that exploits the vector $\bf x$ in an appropriate way.

The graphs in books tend to be univariate input (not always) just because it's straightforward to see what's going on. This is an example of one draw from a GP where the inputs are coordinates $\bf{x}$$=(x_1,x_2)$ on a $30 \times 30$ grid with a kernel I hoped would induce a spatial relationship.

enter image description here

I was reading Bishop 2006 at the time and found it helpful if you want more on the theory - which covers the regression also.

$\endgroup$
3
  • $\begingroup$ Could you provide the actual link to Bishop 2006? I'm unable to find this resource. Could you give some pointers on what adaptations are necessary for multi-input GP? $\endgroup$
    – Yu Chen
    Commented Aug 26, 2017 at 23:21
  • $\begingroup$ The full citation for the book is: Bishop, Christopher M. Pattern recognition and machine learning. Springer, 2006. There is probably a pdf floating around somewhere. You might also want to look at the site: gaussianprocess.org/gpml which has more info online. $\endgroup$ Commented Aug 29, 2017 at 10:59
  • $\begingroup$ Would you happen to be able to share a test example of code in Python that produces the plot above? $\endgroup$
    – Mathews24
    Commented Nov 22, 2018 at 6:02
0
$\begingroup$

There are few answers on StackOverflow that helped me. I think you are looking for anisotropic kernels. They take in inputs with multiple dimensions. They also have multiple length scales for each dimension. Here is an image of the RBF kernel source code for sklearn's implementation.

Sklearn implementation

Sklearn

Here are some similar questions from StackOverFlow:

Stack 1

Stack 2

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.