I am looking for regression techniques which are similar to Kriging/Gaussian process regression, in that no explicit model needs to be specified. (Discounting the prior over functions) I have three independent variables and one dependent variable to which I want to apply such a procedure. The independent variables specify coordinates (locations in 3D), while the dependent variable specifies Wi-Fi signal strength at the given coordinates. Since it is hard to appropriately visualize such high dimensional data, techniques without explicit model dependence are of primary interest. The only similar technique I found was the somewhat unsophisticated Nearest-neighbour interpolation.

Are the above-mentioned techniques the only choices for such a problem?


I would add a more detailed description of the problem you are trying to solve as your question is very vague. What are you trying to attack with Kriging/Gaussian process regression? Any clue on the nature of your problem could really help you get a better answer. Anyway, you could basically use some of a wide array of non-parametric machine learning algorithms. For example: CART, random forest, boosted regression trees, etc. Even though you don't want to specify an explicit model, you do have a problem of the form: $Y:=X_{1}+X_{2}+X_{3}$. You can fit any of these models to your data, for example, fitting random forest in R would go something like this:

fittedmodel <- randomForest(Y~X1+X2+X3,data=yourdata, ntree=1000)

fittedmodel <- randomForest(Y=yourdata[,1],X=yourdata[,2:4], ntree=1000)

The random forest R implementation has more parameters to play with, but in general is a model that requires little tuning so just using the defaults as I am doing here can give acceptable results.

What are you doing this for? if you want to interpolate a map, you can then use this trained model on your unlabelled observations, that simply must have values for the independent variables ($X_{1}, X_{2}, X_{3}$):

interpolation <- predict(fittedmodel,unlabeleddata)

Also, if you are interpolating a map, introducing the coordinates as independent variables is sometimes quite helpful.

| cite | improve this answer | |
  • $\begingroup$ Thanks for the answer; I have added a line about my problem. It is basically about Wi-Fi strengths observed at different locations. $\endgroup$ – Comp_Warrior Aug 19 '13 at 21:38
  • $\begingroup$ Then I definetly think this could definetly be an approach you would follow.Just train one of these models on the locations where you know the Wi-Fi strengh and predict for the rest of the locations. You might get a good model. I don't think it is all that hard to visualize this result (X1,X2,X3) would be your 3d coordinates and the Y could be a color, like a 3d contour plot. There should be some 3d kriging examples out there. In the worst case, something like this: stackoverflow.com/questions/3786189/r-4d-plot-x-y-z-colours. $\endgroup$ – JEquihua Aug 19 '13 at 23:25
  • $\begingroup$ Thanks for the guidance. When I said visualization is hard, I meant to say that I found it hard to ascertain an appropriate model from the kind of visualization you suggested; the Wi-Fi signal strengths are quite close. $\endgroup$ – Comp_Warrior Aug 20 '13 at 8:25

You could try Locally Weighted Linear Regression (wiki article). In fact there is a connection between the two techniques (local regression and GP regression) as described in pg. 26 of the GPML book.

| cite | improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.