In order to deal with complete separation in my data someone suggested that I run penalized GEE (PGEE) by adding a Firth-type penalty term in R.

Although I have read many papers on the Firth correction, I am still not feeling comfortable that I completely understand the concept of Firth correction, as those papers require a much deeper statistical background than I have.

Is there any way to explain this in plain language?


In the ideal scenario, where your data wouldn't be plagued by complete or quasi separation, you would estimate the effects of interest in your binary logistic regression model using maximum likelihood estimation (MLE).

If your sample size $n$ is small or moderate, it turns out that the MLE estimators of these effects suffer from bias under the ideal scenario, with the magnitude of the bias depending on $n$.

As explained in https://bmcmedresmethodol.biomedcentral.com/articles/10.1186/1471-2288-9-56, this bias can be expressed as the additive sum of bias components which are proportional to $1/n$, $1/n^2$, etc. The bias component which is proportional to the inverse of the sample size (i.e., $1/n$) is called 'first order' bias component. (As the sample size increases, the bias converges to 0.)

Firth's method was originally devised to remove first order bias in the MLE estimators of the effects of interest. However, it turns out that it also works well for scenarios where complete or quasi separation is present in the data, producing finite estimators. In that sense, the method produces bias-adjusted estimators. These estimators are simply the MLE estimators from which the (appropriately estimated) 'first order' bias component was subtracted. Firth's method achieves the bias correction by a suitable modification of the so-called score function.

As explained for example at http://prema.mf.uni-lj.si/files/2015-11%20Bordeaux%20Penalized%20likelihood%20Logreg%20rare%20events_e19.pdf, for a 2-by-2 table (binary logistic regression with one binary predictor), Firth’s bias correction amounts to adding 1/2 to each cell of the table.

  • $\begingroup$ Do you know of a guide for when to apply Firth’s method? Is the idea that for small samples or where there is quasi-separation it should be used. For large samples without quasi-separation it should not be used (although if it were the effects of doing so would be negligible as the sample size is in the denominator of the order size). I am thinking the technique is under-utilised, or is this logic too simplistic? $\endgroup$ – Single Malt Nov 9 '20 at 15:13

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.