# Gaussian Process Regression With Multiple Inputs?

Is it possible to use a Gaussian Process to relate multiple independent input variables (X1, X2, X3) to an output variable (Y)?

More specifically, I would like to produce a regression graph like the example shown below where confidence interval reduces around clusters of data (i.e. variance is high at x = 1 where there is no data, but x = 0.3 the regression is tight due to the clustering of input variables) and Instead of having one input variable on the x-axis, where would be multiple inputs.

For example, is it possible to develop a regression relationship that relates the price of houses (HPrice = [125000, 63000, 500000]) to Floor Area (FArea = [856,497,1300]) and the Number of Bedrooms (BedR = [2,2,4])?

Ideally I would like to do this in R, and wonder if there are any recommendations/example available?

Thanks!

• Could you please explain what you have in mind by "using a Gaussian process" and how that would differ from multiple regression?
– whuber
Commented Feb 27, 2014 at 18:58
• Thanks @whuber - I've updated my post so that it is more specific. Commented Feb 27, 2014 at 19:42
• I have the same question, all the examples are just show the same stupid example, which is y=sin(x) + N() Commented Aug 4, 2021 at 4:40

Yes, it is entirely possible. Recall the Gaussian Process model is defined by a kernel function, $K(\bf x,\bf x')$, yours is a case where you need some function that exploits the vector $\bf x$ in an appropriate way.

The graphs in books tend to be univariate input (not always) just because it's straightforward to see what's going on. This is an example of one draw from a GP where the inputs are coordinates $\bf{x}$$=(x_1,x_2)$ on a $30 \times 30$ grid with a kernel I hoped would induce a spatial relationship.

I was reading Bishop 2006 at the time and found it helpful if you want more on the theory - which covers the regression also.

• Could you provide the actual link to Bishop 2006? I'm unable to find this resource. Could you give some pointers on what adaptations are necessary for multi-input GP? Commented Aug 26, 2017 at 23:21
• The full citation for the book is: Bishop, Christopher M. Pattern recognition and machine learning. Springer, 2006. There is probably a pdf floating around somewhere. You might also want to look at the site: gaussianprocess.org/gpml which has more info online. Commented Aug 29, 2017 at 10:59
• Would you happen to be able to share a test example of code in Python that produces the plot above? Commented Nov 22, 2018 at 6:02

There are few answers on StackOverflow that helped me. I think you are looking for anisotropic kernels. They take in inputs with multiple dimensions. They also have multiple length scales for each dimension. Here is an image of the RBF kernel source code for sklearn's implementation.

Sklearn

Here are some similar questions from StackOverFlow:

Stack 1

Stack 2