# How many experiments to run (sample-size) if I know I am going to feed them to a non-parametric regression?

I have 2 input variables, $$X_1$$ and $$X_2$$ that affect output variable $$Y$$. I can run experiments where I modify the inputs and measure what happens to the output.

Now, if $$X_1$$ and $$X_2$$ were binary, I could just run four sets of experiments, collect $$Y$$ and use t-test to figure out if any combination affects output differently. Most importantly i can use power tables to check what should the sample size for each experiment be in order to achieve a desired power level.

However, if $$X_1$$ and $$X_2$$ are continuous between 0 and 1 I am not sure how to compute sample size and power. I am willing to assume some form of continuous function connecting $$Y$$ to $$f(X_1,X_2)$$ although not necessarily a linear one, but I am unclear about what my power is when I check, for example, if the difference between $$f(X_1,X_2)$$ and $$f(X_1,X_2+h)$$ is significant or not.
And given this, can I figure out what the sample size ought to be?

• Exactly what are you trying to accomplish? It sounds something like response surface modeling, but then power is not a principal concern. – whuber Oct 17 '19 at 11:42
• Well at the end of the day I want to know how sure I am whether f(x) and f(x') are different (for arbitrary x, x'). I want to run just enough simulations (probably sampling at random uniformly around x, x') to achieve a given level of confidence – CarrKnight Oct 17 '19 at 13:14
• Your question is ill-defined: what would "arbitrary x, x'" possibly mean? If randomly chosen, sometimes they will be so close that no amount of sampling could possibly distinguish the responses. – whuber Oct 17 '19 at 13:34
• Exactly! is there a function/table connecting distance between X and sample size needed to be sure the response are different when the regression says they are? – CarrKnight Oct 17 '19 at 15:07
• When you do the experiment with just two values of $X,$ yes there is such a function. In general there is not: it depends on all the values of $X$ you use, on the nature of $f,$ and even on the experimental design (e.g., whether it is adaptive). You can't figure out sample size until you have specified the experiment in full detail. – whuber Oct 17 '19 at 15:20