1
$\begingroup$

I have a data set where my binary dependent variable is rarely equals one (about 0.3 % of all observations). My goal is to predict the dependent variable based on a few variables and a constant term. I was wondering whether a penalized logit regression approach could make up for the fact that I have only a few events. I know that penalized regression leads ultimately to variable selection; variable selection is not my goal though.

The reason why I thought that penalized regression my help is based on the following observation. Since the number of events is small, the estimate of the slope coefficient in the Logit regression is very large in magnitude as a large slope leads to predicted probabilities close to zero (or close to one, depending on the sign of the variable). Since the penality term causes the estimated coefficients to be shrunk towards zero, I thought that using penalized regression may be helpful in sitation with only a few events. Is this line of thought reasonable?

Edit (Some more details about what I do). I want to predict the probability of complications after a particular surgery based on patient-specific characteristics and a disease-related score. The complications occur rarely. The hope is that the score helps predicting complications. The results so far (from a non-penalized logistic regression) suggest that the score does not help at all. This can be of course the case. However, my feeling is that score is not as effective as expected because there are only a few patients with complications. This motivated our idea to use a penalized regression approach to account for the low number of events.

Improve this question
$\endgroup$
5
  • $\begingroup$ A rare event will more likely yield a small intercept parameter than a large negative slope. Penalized models are usually a Good Thing, though. Have you already fitted a non-penalized model? $\endgroup$
    – Stephan Kolassa
    Commented Mar 4, 2022 at 17:56
  • $\begingroup$ Thank you for your reply. You are totally right! I made a mistake in my reasoning. Of course, the intercept should be large so that probabilities close to zero (or one) are achieved no matter what the values of the explanatory variables are. Do you have literature recommendations for penalized models with rare events? I tried a non-penalized model, but the results are not in line in what we expected (and I assumed that is because of the low number of events). $\endgroup$
    – Syd Amerikaner
    Commented Mar 4, 2022 at 20:19
  • $\begingroup$ Unfortunately, I don't have any literature I could recommend. Logistic regression, and especially the penalized sort, usually does not have a problem with rare events, so if your results are unexpected, this may be due to other causes. Perhaps you can elaborate on your model and what was unexpected about the results? $\endgroup$
    – Stephan Kolassa
    Commented Mar 4, 2022 at 20:24
  • $\begingroup$ I added a little bit of context $\endgroup$
    – Syd Amerikaner
    Commented Mar 5, 2022 at 12:02
  • 1
    $\begingroup$ My first suspicion would be that the score depends on the patient characteristics, e.g., obese patients have worse scores, you record both obesity and the score, and it turns out that the score does not explain complications any better after the model already includes obesity. Are you looking at that? I'm not saying penalization is a bad idea, it's usually good, just that there may simply not be a lot to find in your data. And yes, rare events simply do not yield a lot of information. This may be helpful. $\endgroup$
    – Stephan Kolassa
    Commented Mar 5, 2022 at 15:09

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.