I have data sample size of almost 15,000 cases. The dependent variable is a dichotomous variable stating whether the patient has the disease or not, Yes=1, and No=0. I have 12 more independent variables which are continuous as well as dichotomous. My question here is, before I apply the logistic regression, I need to know which parameters have a great impact on the DV; how do I do that? Secondly, there are 7 different methods of using binary logistic regression like enter, forward conditional, forward LR, etc; what is the basic difference between these? I couldn't find any matter on this on the internet.
$\begingroup$
$\endgroup$
7
-
1$\begingroup$ You don't need to know beforehand which have a great impact on the dependent variable - that's what the regression's for. How many "Yes"s have you? Too few compared to the degrees of freedom for independent variables & there's danger of over-fitting. $\endgroup$– Scortchi - Reinstate Monica ♦Commented Aug 5, 2014 at 10:57
-
$\begingroup$ @Scortchi Thanks for your reply, I have 1240 yes's and 13527 No's, shall I reduce my sample size to equal no. of Yes's n No's? Please help, i really need some guidance $\endgroup$– ZehraCommented Aug 6, 2014 at 1:51
-
1$\begingroup$ (1) The usual rules of thumb suggest 10 - 20 observations in the minority class (here that's "Yes"s) per degree of freedom for the regression should avoid serious overfitting. Unless you've lots of interactions or non-linear effects you're well within that guideline. Fit the full model & validate it. (2) If you've already gone to the trouble of measuring the independent variables on all 14,767 then don't throw any data away. If you haven't then (depending on how troublesome it will be) you might want to knock off after 2 to 6 thousand or so "No"s. $\endgroup$– Scortchi - Reinstate Monica ♦Commented Aug 6, 2014 at 9:07
-
1$\begingroup$ (3) I'd guess you're using SPSS: "Enter" refers to fitting a model with the independent variables you specify; the other options carry out variable selection by stepwise methods. I'd strongly suggest not carrying out any kind of automatic variable selection based on relationship with the dependent variable unless you have a clear idea what you want to get out of it, & know how to check whether you got that or not, as well as understanding the disadvantages. $\endgroup$– Scortchi - Reinstate Monica ♦Commented Aug 6, 2014 at 9:39
-
2$\begingroup$ @Scortchi Consider answering this question with your comments so it counts as answered "(When shouldn't I comment?: Answering a question or providing an alternate solution to an existing answer; instead, post an actual answer (or edit to expand an existing one)" :) $\endgroup$– FirebugCommented Jan 29, 2018 at 15:30
|
Show 2 more comments
1 Answer
$\begingroup$
$\endgroup$
The methods of "doing the regression" that you refer to (with SPSS terminology) are really methods of variable selection. You shouldn't do any kind of automatic variable selection. You should just use all your candidate variables in the model, and let them stay there. Using data for variable selection
does have a very poor track record of actually finding the correct variables,
invalidates posterior inference. Very thorough discussion.
The regression model is what tells you which variables are important and which not. For logistic regression it is often advised that with the effective number of observations $N$ (number of obs in the minority class), $N$ should be at least 15 times the number of parameters. More details here. With 15000 observations that should not be a problem for you, even with only 10% in the minority class. With 1% in the minority class you could have problems. Just fit the full model and validate it!
You say you have some non-linearities. That can be solved by representing variables who acts nonlinearly with splines. For instance, I would always represent age via splines. See also the useful information in the comments.
answered Oct 4, 2018 at 17:10