Sample point locations and multiple linear regression

I have a question on multiple polynomial regression and the absolute minimum amount of points in the different terms. The minimum amount of points required for a second order polynomial would (in one variable) be three and in general it would p+1, p being the polynomial order. I have the intuition this generalizes to more than one variable but I cannot prove it and I would like to inspect my design matrix term by term (thus column by column) for sufficient amounts of points. For example take the following model:

$y = b_0 + b_1x_1 + b_2x_2 + b_{12}x_1x_2$

The order of the last term is 2, suggesting three points are needed to sufficiently vary this term. Can anyone guide me to a proof that the third column in my design matrix should have at least three distinct values?

Another way of stating the question would be: would one run into trouble sampling the points $x_1$ and $x_2$ on the hyperbola $x_1x_2=\mathrm{const}_1$ (or on $x_1x_2=\mathrm{const}_1$ and $x_1x_2=\mathrm{const}_2$?

• This seems like a math question, not a statistics one. In statistics, we would want (indeed, require) many more points than the mathematical minimum. There's nothing really about regression here - it's more the minimum points to define certain curves in various spaces. – Peter Flom Dec 14 '12 at 11:27
• Even in statistics one should inspect the design matrix for adequacy, apart from doing the hypothesis testing on coefficients and VIF analysis... – Murphy Dec 14 '12 at 12:19
• I have merged your two unregistered accounts, Murphy. Please, don't forget to register. – chl Dec 14 '12 at 12:57

1 Answer

If you're just interested in that model (so interaction term with no quadratic terms) you need $2^k$ points, were $k$ is the number of variables. In your case, with 2 variables you need 4 points. Of course these points should be chosen on logical basis in order to avoid auto-correlation of the predictors. Sampling your points on a regular grid and coding your variables it will produce a model with VIF on each coefficients of the model exactly equal to 1.

A really good tip would be to choose orthogonal variables, for example:

    x1<-c(1,-1,1,-1)
x2<-c(-1,-1,1,1)


If you're interested on this topic I suggest you to read something about experimental designs and response surface methodology.