4
$\begingroup$

I've been performing some logistic regressions and found some great results. The problem is I'm unsure if my model is stable or not. This is mostly because I am confused about what people mean by cell size for logistic regression.

I keep reading inconsistent prescriptions. Some people say I need 10-15 observations per variable while some people say I need 10-15 observations per cell. I also get confused if people mean per cell when including all variables or just the main exposure/control vs event/ no event.

The overall N is 331.

Let's say I have data in a 2x2 table with the typical exposure/control by event/ no event:

.                        Event       No Event
.
.     Exposure             103          15
.
.  No Exposure             150          63

In this example, the smallest cell size would be 15. So, I could perform a logistic regression with 1 independent variable, right?

So let's say I wanted to add another variable to the same data, like Race (African American or not, 1 or 0) and here's the cell breakdown:

.     Clinic      Event    Race    N
.       0           1        1     121
.       1           1        1     92
.       0           0        1     46
.       0           1        0     29
.       0           0        0     17
.       1           0        1     14
.       1           1        0     11
.       1           0        0      1

Would this now mean that my smallest cell is 1? Or, would I still refer to the original 2x2 table and it would still be 15?

I'm asking because when I run my logistic model in SAS with 6 independent variables on this same data, I do not get any small cell warnings and the convergence criteria is met. Does this mean I'm okay and don't need to necessarily worry about cell size?

I have been running exact logistic regression models anyway but they take a VERY long time (whole other story). I'd rather just report the standard LR results to save time (assuming they are legitimate).

$\endgroup$
  • $\begingroup$ Could you explain what you mean by "stable"? Ordinarily one might think that refers to the standard errors of parameter estimates being satisfactorily small, but apparently not here--for otherwise you would just consult their values in the output. What sources are you referring to with the rules of thumb you are quoting? $\endgroup$ – whuber May 27 '16 at 20:34
  • $\begingroup$ I suspect there are several different things to worry about: 1) are estimates normally distributed so that est & SE makes sense, 2) is there a concern about overfitting, because a complex model is fitted on very little data. Plus perhaps something else? $\endgroup$ – Björn May 28 '16 at 5:20
  • $\begingroup$ @Björn 1) Yes I need to check the standard error estimates. If they are normal, things should be okay right? 2) Yes my concern is overfitting. Long story short, I just want to make sure I'm using the method correctly. I've done many LR's before but not many with small cell sizes. $\endgroup$ – Pete May 30 '16 at 16:25
  • $\begingroup$ @whuber Stable meaning that I don't have parameter estimates and confidence intervals that are gigantically huge (like an OR of 45 with 95% CI 25 - 65). I'm not at work where I have the source material handy, but here's one that I found just to give at least some sort of reasoning: citeulike.org/user/harrelfe/article/13467382 $\endgroup$ – Pete May 30 '16 at 16:29
  • $\begingroup$ Since (presumably) you do have CIs or standard errors for your parameter estimates, why then are you "unsure" about the stability of those estimates? $\endgroup$ – whuber May 30 '16 at 17:44
1
$\begingroup$

When they say "per Cell", generally in my experience people mean per cell in the larger NxM table created by adding your covariates.

This is however a very rough and informal guideline - as you're finding, logistic regression is fairly mechanically reliable, and can be made to do a great many things it probably shouldn't before running aground on convergence issues. For example, having one cell that violates that rule might not be a big deal, whereas having many may very well create problems.

There are two hazards with what you're currently doing:

  1. The model is essentially being asked to smooth over places where there isn't very much data, which carries the risk of missing things that exist in your exposure-disease relationship.
  2. It's possible (though not a problem in your example) that a cell size of zero is not merely due to stratification, but because something is impossible, and the regression model will also miss that.
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.