# Regression techniques similar to Kriging/Gaussian process regression

I am looking for regression techniques which are similar to Kriging/Gaussian process regression, in that no explicit model needs to be specified. (Discounting the prior over functions) I have three independent variables and one dependent variable to which I want to apply such a procedure. The independent variables specify coordinates (locations in 3D), while the dependent variable specifies Wi-Fi signal strength at the given coordinates. Since it is hard to appropriately visualize such high dimensional data, techniques without explicit model dependence are of primary interest. The only similar technique I found was the somewhat unsophisticated Nearest-neighbour interpolation.

Are the above-mentioned techniques the only choices for such a problem?

I would add a more detailed description of the problem you are trying to solve as your question is very vague. What are you trying to attack with Kriging/Gaussian process regression? Any clue on the nature of your problem could really help you get a better answer. Anyway, you could basically use some of a wide array of non-parametric machine learning algorithms. For example: CART, random forest, boosted regression trees, etc. Even though you don't want to specify an explicit model, you do have a problem of the form: $Y:=X_{1}+X_{2}+X_{3}$. You can fit any of these models to your data, for example, fitting random forest in R would go something like this:

fittedmodel <- randomForest(Y~X1+X2+X3,data=yourdata, ntree=1000)

fittedmodel <- randomForest(Y=yourdata[,1],X=yourdata[,2:4], ntree=1000)


The random forest R implementation has more parameters to play with, but in general is a model that requires little tuning so just using the defaults as I am doing here can give acceptable results.

What are you doing this for? if you want to interpolate a map, you can then use this trained model on your unlabelled observations, that simply must have values for the independent variables ($X_{1}, X_{2}, X_{3}$):

interpolation <- predict(fittedmodel,unlabeleddata)


Also, if you are interpolating a map, introducing the coordinates as independent variables is sometimes quite helpful.

• Thanks for the answer; I have added a line about my problem. It is basically about Wi-Fi strengths observed at different locations. – Comp_Warrior Aug 19 '13 at 21:38
• Then I definetly think this could definetly be an approach you would follow.Just train one of these models on the locations where you know the Wi-Fi strengh and predict for the rest of the locations. You might get a good model. I don't think it is all that hard to visualize this result (X1,X2,X3) would be your 3d coordinates and the Y could be a color, like a 3d contour plot. There should be some 3d kriging examples out there. In the worst case, something like this: stackoverflow.com/questions/3786189/r-4d-plot-x-y-z-colours. – JEquihua Aug 19 '13 at 23:25
• Thanks for the guidance. When I said visualization is hard, I meant to say that I found it hard to ascertain an appropriate model from the kind of visualization you suggested; the Wi-Fi signal strengths are quite close. – Comp_Warrior Aug 20 '13 at 8:25

You could try Locally Weighted Linear Regression (wiki article). In fact there is a connection between the two techniques (local regression and GP regression) as described in pg. 26 of the GPML book.