4
$\begingroup$

When estimating parameters, I know the general rule of thumb is n/parameters should be >10. Does the intercept in a model count as one of the estimated parameters in this "rule"?

For example, if I had an n of 50, would I only be able to estimate 4 parameters, as the intercept would make up the fifth?

$\endgroup$
1

2 Answers 2

5
$\begingroup$

It's a rule of thumb, as you say.

I'd put it whimsically: the intercept doesn't know it's different, so its estimates feel free to misbehave. So yes, the intercept is included. Even more so, the intercept doesn't know if you don't care about it or regard it as uninteresting.

I'd turn it round and ask for more information on your concerns. One thing I've often seen is people estimating an intercept that is far outside the data when re-parameterisation would give a better view. For example, people fit regressions to recent data with time in years as the predictor. Hence the intercept is the level at year 0 and is rarely well determined in that situation. Recasting the model to predicting from (year - 2000), or whatever, means that the intercept is the level at year 2000. It's more interesting and its uncertainty will look more sensible.

Naturally, this may be a long way from what you are doing, but the definition of intercept is more a matter of convention than people often realise.

$\endgroup$
4
$\begingroup$

I would like to find a truly general formula for the OLS case, but a couple of observations: Start with $n$ to well-estimate the intercept, e.g., $n=100$, then add $k$ observations per estimated or entertained parameter. When the signal:noise ratio is low, simulations I have done point to $k=15$. I'd like to have a formula that takes into account the signal:noise ratio (e.g., true $R^2$). When this ratio is high you can estimate more parameters.

But we need to decide on the metric for model performance before answering the question. I don't think that power is the proper metric. What I prefer, and what the above is based on, is finding $n$ such that the apparent ordinary $R^2$ for the model is an unbiased estimate of the true long-run model performance.

I'm going to expand on this at Rules of thumb for minimum sample size for multiple regression

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.