2
$\begingroup$

I have a question on multiple polynomial regression and the absolute minimum amount of points in the different terms. The minimum amount of points required for a second order polynomial would (in one variable) be three and in general it would p+1, p being the polynomial order. I have the intuition this generalizes to more than one variable but I cannot prove it and I would like to inspect my design matrix term by term (thus column by column) for sufficient amounts of points. For example take the following model:

$y = b_0 + b_1x_1 + b_2x_2 + b_{12}x_1x_2$

The order of the last term is 2, suggesting three points are needed to sufficiently vary this term. Can anyone guide me to a proof that the third column in my design matrix should have at least three distinct values?

Another way of stating the question would be: would one run into trouble sampling the points $x_1$ and $x_2$ on the hyperbola $x_1x_2=\mathrm{const}_1$ (or on $x_1x_2=\mathrm{const}_1$ and $x_1x_2=\mathrm{const}_2$?

$\endgroup$
3
  • 1
    $\begingroup$ This seems like a math question, not a statistics one. In statistics, we would want (indeed, require) many more points than the mathematical minimum. There's nothing really about regression here - it's more the minimum points to define certain curves in various spaces. $\endgroup$ – Peter Flom Dec 14 '12 at 11:27
  • $\begingroup$ Even in statistics one should inspect the design matrix for adequacy, apart from doing the hypothesis testing on coefficients and VIF analysis... $\endgroup$ – Murphy Dec 14 '12 at 12:19
  • $\begingroup$ I have merged your two unregistered accounts, Murphy. Please, don't forget to register. $\endgroup$ – chl Dec 14 '12 at 12:57
1
$\begingroup$

If you're just interested in that model (so interaction term with no quadratic terms) you need $2^k$ points, were $k$ is the number of variables. In your case, with 2 variables you need 4 points. Of course these points should be chosen on logical basis in order to avoid auto-correlation of the predictors. Sampling your points on a regular grid and coding your variables it will produce a model with VIF on each coefficients of the model exactly equal to 1.

A really good tip would be to choose orthogonal variables, for example:

    x1<-c(1,-1,1,-1)
    x2<-c(-1,-1,1,1)

If you're interested on this topic I suggest you to read something about experimental designs and response surface methodology.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.