1
$\begingroup$

In a statistical model e.g. regression, we have to ensure the sample size is sufficient to estimate a given number of parameters. Rules of thumb e.g. n=10 per parameter, or a power analysis, will give us the total sample size needed to fit the model. However, I was wondering how these sample size estimates apply to categorical variables which are usually coded as dummy variables, with a parameter estimated for 1 less than the number of groups (factor levels). Supposing we have a sample size which, on the average, has 30 data points for each parameter to be estimated (overall ratio = 30:1). However a categorical variable has one factor level with a very "small" number of occurances e.g. 4, giving a ratio = 4:1. This is much less than the overall ratio and even less than the rule of thumb of 10:1. I was wondering how to best proceed in such circumstances? Would it matter so much that the sample size / number parameters ratio is very small in one factor level if overall the ratio is sufficient?

$\endgroup$

1 Answer 1

1
$\begingroup$

The 10:1 rule of thumb is highly approximate and doesn't include the sample size needed to just estimate the intercept. For example in binary logistic regression you need n=96 just to estimate th e intercept (i.e., when there are no predictors needing to be modeled). If you have a single categorical predictor with 3 levels you'd need at least n=96 in each of the levels.

$\endgroup$
5
  • $\begingroup$ To clarify, are you saying that the minimum sample size per parameter (assuming we have calculated this correctly) should be applied to EACH parameter (including all dummy variables) to be estimated rather than as an average ratio across the entire data set to be modelled? $\endgroup$
    – user167591
    Commented Mar 26 at 13:51
  • $\begingroup$ Your example sounds a bit extreme. Would it not be a baseline minimum of n = 96 to estimate intercept with 10% MOE, and then plus x for each additional parameter estimated? Suppose x was calculated as 15, would it not be n = 96+15+15 = 126 required for categorical predictor with 3 levels? $\endgroup$
    – user167591
    Commented Mar 27 at 1:53
  • 1
    $\begingroup$ That's roughly correct but a much better approach is by Richard Riley et al. Here's the paper for continuous Y: onlinelibrary.wiley.com/doi/abs/10.1002/sim.7993 for which there is an R app to help. $\endgroup$ Commented Mar 27 at 11:22
  • $\begingroup$ Thanks @Frank Harrell. Is there an argument that we can still run a multiple regression model with one or more categorical predictors having low counts in one or more levels, and just let the confidence interval reflect uncertainty in our estimates? I think that would work if we apply shrinkage e.g. mixed effects model to help avoid overfitting? $\endgroup$
    – user167591
    Commented Mar 28 at 4:19
  • 1
    $\begingroup$ You can use confidence intervals even without shrinkage. I would suppress p-values. $\endgroup$ Commented Mar 28 at 20:58

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.