2
$\begingroup$

I am new to R programming and although I searched through the community, I couldn't find a similar topic, although it has to be somewhere. So a link to a similar case would be sufficient.

I have a data set of 110 cases and about 50 variables. Most of the variables are binary, some categorical or numeric. All of them could be made binary if needed. The 50 variables represents factors in written reports that should predict the decision if an action is taken (1) or not (0).

I want to perform a retrospective study and see which variables were the key factors in decision making. I have performed a pearson chi square test on every single variable and only 2 correlate significantly with action or not. Now I want to calculate which combination of variables were needed for an action. I only don't know which test I have to perform for this.

Anyone has an idea how to test this in R

Improve this question
$\endgroup$

2 Answers 2

Reset to default
2
$\begingroup$

If you don't use a massive penalty as with lasso you are asking for all kinds of problems. Think about it this way: it requires 96 observations just to estimate the intercept in the logistic model, with no covariates at all, and that only gets you a margin of error of +/- 0.1 in estimating the overall probability of outcome. How much more difficult it is to estimate conditional probabilities as a function of the covariates/predictors. Personally I think your project is hopeless.

Improve this answer
$\endgroup$
6
  • 2
    $\begingroup$ "it requires 96 observations just to estimate the intercept in the logistic model" — Given that an intercept-only model could be identifiable with as few as two observations, what do you mean by this? How did you arrive at the oddly precise number 96? $\endgroup$
    – Kodiologist
    Jul 10, 2017 at 16:40
  • $\begingroup$ You can get the maximum likelihood estimate of the intercept with only one observation. That's not the point. The issue is the reliability/precision of estimates. If you were to over-analyze a dataset to get estimates, tests, and rank predictors, the result may be meaningless. Please do not deduct points from an answer unless you fully understand the answer. To see the derivation of the 96 see section 5.6.3 of my BBR notes for which the link is at fharrell.com/p/blog-page.html . This is discussed in the context of logistic regression in my Regression Modeling Strategies text $\endgroup$
    – Frank Harrell
    Jul 10, 2017 at 19:09
  • $\begingroup$ Thank you, Frank Harrel, for your answer. If I increase the cases to 200 and pick the top 10-15 correlations, would the results be more reliable? $\endgroup$
    – Tim Withag
    Jul 10, 2017 at 19:28
  • $\begingroup$ I wasn't the one who downvoted this answer. I find the section you pointed me to telegraphic, so I'm not sure if I understand it correctly. In any case, I agree that precision (and accuracy) matter, but there is no magic threshhold between precise and imprecise. Rather, the number 96 arises because you happened to desire 95% confidence of absolute error within .1 (and I think you also have an assumption that $p$ isn't too extreme). So it would be more precise to say that, rather than to say that you need 96 observations to estimate the value at all. $\endgroup$
    – Kodiologist
    Jul 10, 2017 at 19:32
  • 1
    $\begingroup$ That's a good summary. By estimating the # needed to get a +/- 0.01 margin on prob you are also talking about the precision of $\beta_{0}$. But it's hard to escape the notion that if the original sample is large enough only to estimate the intercept, it is unlikely to be large enough to model predictor effects. The 96 comes from the worst case of $p=0.5$. $\endgroup$
    – Frank Harrell
    Jul 10, 2017 at 19:35
0
$\begingroup$

There are combinatorial number of possibilities when you test the variable importance / correlations in the way you described. And it is not really feasible.

In stead, you may try logistic regression with LASSO penalty instead. My answers here gives some details.

Regularization methods for logistic regression

And this post explains Lasso well.

Why does the Lasso provide Variable Selection?

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.