# How does required sample size scale with the number of features in multiple linear regression?

Assume a multiple linear regression model with $F$ features and $N$ samples. For each feature, a standard error on the $\beta$ coefficient and a Student's T P-value can be calculated. The P-value tests the null that the $\beta$ coefficient is zero against the alternative that the $\beta$ coefficient is nonzero.

If I want to maintain a fixed standard error on the $\beta$ coefficients and a fixed power in the Student's T tests, how does the required $N$ scale with increasing $F$, in terms of big-O notation? Similarly, what would happen to my standard errors and statistical power if I doubled both $N$ and $F$?

• That doesn't seem to me to be a sufficiently well-defined problem; for example, if you add one feature, but it's highly informative w.r.t. the residuals of the "current" model, the std. errors on the existing coefficients will go down, perhaps substantially, even with no increase in sample size; if the additional feature is sufficiently close to orthogonal to the residuals, it may increase the std. errors of the existing coefficients. – jbowman Dec 23 '11 at 23:23

Suppose there were no covariates, so that the only parameter in the model were the intercept. What is the sample size required to allow the estimate of the intercept to be precise enough so that the predicted probability is within 0.1 of the true probability with 0.95 confidence, when the true intercept is in the neighborhood of zero? The answer is n=96. What if there were one covariate, and it was binary with a prevalence of $\frac{1}{2}$? One would need 96 subjects with $X=0$ and 96 with $X=1$ to have an upper bound on the margin of error for estimating $\textrm{Prob}\{Y=1|X=x\}$ not exceed 0.1 for either value of $x$. The general formula for the sample size required to achieve a margin of error of $\delta$ in estimating a true probability of $\theta$ at the 0.95 confidence level is $n = (\frac{1.96}{\delta})^{2} \times \theta(1 - \theta)$. Set $\theta = \frac{1}{2}$ (intercept=0) for the worst case.
This idea can be extended by simulation of a number of continuous covariates or by imagining the effective number of unique orthogonal "groups" there are in the covariate space for which one desires good prediction accuracy. $96\times k$ where $k$ is the number of "groups" would be an estimate of the required sample size.