3
$\begingroup$

I am dealing with logistic regression, trying to identify variables which have a causal relationship with a binary response. The way I usually do it is to try variables one by one and visualize the probability of positive outcome curve, and check if it is flat or has a good curve. If it is the latter, then it means there's a causal relationship.

I wonder if there is a better routine? Especially if I have a huge number of variables to check, while some of them are not the ones that have a relationship with the observations. What would be the cons if I throw too many variables into logistic regression?

$\endgroup$
7
$\begingroup$

I would recommend trying lasso regression. The glmnet package for R is great.

Lasso regression is nice because it variable selection is incorporated into fitting the model, which is much better than the ad-hoc technique you describe above.

| cite | improve this answer | |
$\endgroup$
3
$\begingroup$

I am not too happy with your approach.

First, the causal effect in your data is not identified. All you do is look for correlations, but these need not imply a causal relationship. It may all be caused by an omitted variable. This is further exacerbated by the fact that you check your variables 1 by 1 - then you definitately have omitted variable bias. Hence, not even the correlations you find may be correct: If factor $A$ looks like it has large correlation with the binary response, this may actually be because it is strongly correlated with factor $B$, which itself has a strong correlation with the binary response. This is why one should run those regressions with all relevant variables.

If you have many variables, but not so many that you lack degrees of freedom to compute the covariance matrix, then just run the regression with all factors. The point estimates then give you a pretty good idea about which factors seem to contribute a lot to the outcome, and the standard errors tell you whether factors stray a lot from the estimated model. If you are truly interested in causal effects, then you will have to start thinking about experiments, matching or instrumental variables in an OLS setting.

| cite | improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.