5
$\begingroup$

I would like to point out that I am new to this field, so if I am not clear please forgive me (and correct me).

I set up a DoE (Design of Experiment) with 11 inputs and 121 runs. I used a STOA (Strength-Two Orthogonal Array) to fill the domain and I ran my test. I measured 5 outputs, so at the end I have a 121x11 input array and a 121x5 output array. My task is to generate a RSM (Response Surface Model) from these data and I am currently using the kriging algorithm but I do not get good results in terms of accuracy (the RMSE (Root Mean Square Error) is quite high).

My questions are:

  1. since kriging comes out from geostatistics applications, it is correct to say that it works well just with 2D or 3D spaces and it loose its accuracy with high dimensional spaces? Why or why not?
  2. given my training set, how can I choose the best algorithm to fit my RSM?

EDIT1: @whuber - I got your concerns. I actually have some lack about data analysis and I want to look into it. Where can I start? Do you know some god books or websites?

EDIT2: @GeoMatt22 -

  1. I performed a Leave-One-Out Cross Validation and I plotted the Actual vs Predicted graph. For some outputs the distribution of points is rather spread and the RMSE is high. I suppose I should do some data analysis. What should I look at? Variance of data? Covariance?
  2. Goal is optimization.

EDIT3: @David Kozak - Thanks for your observations. I will take a look to the Rasmussen and Williams.

Improve this question
$\endgroup$
2
  • 2
    $\begingroup$ Your problems might stem, at least in part, from the concept of kriging as an "algorithm." Yes, there is an algorithm involved, just as there are algorithms and formulas involved in any statistical analysis. But what makes kriging work is the preliminary data analysis that is performed, centering on developing an appropriate variogram. I am concerned that if the role of data analysis and variography is not properly appreciated and acknowledged, then answers to your question might be misleading. $\endgroup$
    – whuber ♦
    Commented Feb 6, 2017 at 18:46
  • 3
    $\begingroup$ For 1., there is no inherent problem using Kriging in high-dimensional spaces necessarily, and it is commonly used in surrogate modeling (e.g. Design and Analysis of Computer Experiments). As Kriging is an interpolation algorithm, what is your error? (e.g. some sort of cross-validation?) For 2., what is the goal of your surrogate model? (e.g. optimization? sensitivity?) $\endgroup$
    – GeoMatt22
    Commented Feb 6, 2017 at 20:47

2 Answers 2

Reset to default
2
$\begingroup$

I completely agree with David Kozak's answer, but I would like to bring some additional points:

11 inputs is not too high for Gaussian process regression (GPR)/Kriging. I have already dealt with problems with a dozen of variables with success using GPR

121 observations is indeed quite small, and the second concern about the training dataset is that it has been created with an Orthogonal Array (OA). While they are fell suited for linear regression, they are not for GPR. GPR with stationary kernels (the most common and simple ones) use the distances between points of the training set and need as much diversity in these distances, but OA have only few distances compared to a random design of the same size. When using GPR, it is better use rather Latin Hypercube sampling or random design.

But I guess that you cannot change your design of experiment... As Kriging may give disappointing results, I would advise you to test more simple methods as linear regression, GAM... Random Forests (or more simply a decision tree) might give interesting results too.

Improve this answer
$\endgroup$
1
  • $\begingroup$ Agreed that 11 parameters is not a problem in general. But. With 121 data points the data space will be exceptionally sparse and the uncertainty will be massive. I don't think the problem will be ameliorated by switching to a simpler method -- perhaps some bayesian linear regression if @NewGuest has a sense of what the parameter space ought to look like and can use strong priors. I would expect OLS regression to give completely untrustworthy point estimates with very high uncertainty. Getting more data or reducing the parameter space would still be my go-to. $\endgroup$
    – David Kozak
    Commented Nov 17, 2017 at 23:55
2
$\begingroup$

You might look into Gaussian Process Regression... Kriging is the name used for Gaussian Processes in Spatial Statistics, and is the domain in which they found the most use until relatively recently. There's an excellent book freely available online by Rasmussen and Williams on the topic.

Without knowing more about your problem or your data it would be difficult to suggest where to look as far as off-the-shelf algorithms go. I'd suggest that 121 observations is too few for that many dimensions without a strong prior understanding of what your parameter values ought to be, so perhaps make sure that they're all needed, and try some sort of feature selection or dimension reduction if they're not.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.