5
$\begingroup$

Given a multivariate regression, in a form bellow, what would be the minimum number of observations ($n$)? $$\mathbf{Y}=\mathbf{X}\mathbf{B}+\mathbf{E},$$ where $\mathbf{Y}, \mathbf{X}$ and $\mathbf{E} \in R^{n\times m}$ $(n>m)$ and $\mathbf{B}\in R^{m \times m}$. You can treat each column of $\mathbf{Y}$ and $\mathbf{X}$ as a time series.

Based on Frank Harrell's book, Regression Modeling Strategies, if you expect to be able to detect reasonable-size effects with reasonable power, you need 10-20 observations per parameter (covariate) estimated. I wonder if this would still hold in multivariate regression rather univariate? In the univariate the dependent variable $Y$ is one time series (vector format) and in multivariate one, it consists of more than one time series (matrix format).

For more info on multivariate reg: This starting from p.43.

$\endgroup$

1 Answer 1

6
$\begingroup$

I believe that it is better to think in terms of power to detect effects of interest, rather than rules of thumb. That said, if you want a quick-and-dirty baseline value, the standard rule of thumb is $N=10$ per parameter. Note that in a multivariate context, the number of parameters is greater than the number of covariates. Consider a multiple regression with one $Y$ and three $X$'s: that would be a minimum of $N=30$. However, if there were three $Y$'s, that would be $N=90$.

Note further that this ignores covariance parameters (people often care primarily about the means). I don't know of an analogous rule of thumb for covariances, but if you wanted reasonable estimates of the covariances, they will grow much more rapidly as the number of $Y$'s increases: with $p$ $Y$'s, the variance-covariance matrix will have $^{p(p+1)}/_2$ parameters to estimate for each $X$ variable. Moreover, since these depend on the means already being stable, you should think of those extra tens being added on top of the original requirement. That is, in our three $Y$, three $X$ example, you would need $N = 60$ (per $X$), times three $X$'s, is $N = 180$, plus the original ninety, is $N=270$. That certainly seems like a lot, but if you care about those parameters, there's a lot of them.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.