I am implementing a logistic regression on a 250 x 20 dataset (250 observations of 20 variables) with a dichotomous response. In this proces I have encoutered some different problems, namely:
1. Group sizes
I am aware that small group sizes can be troublesome, although it is not completely clear to me when the problem is to be taking serious. Most of my predictor variables are fairly evenly distributed, but one continuous predictor only has very few observations (1-3) in extreme cases. Moreover, in two categorical predictors taking values 0 and 1 only about 10 % are 1's. Taking the size of the dataset into account, could this be a potential problem?
2. Correlations
There is a high correlation between some of the predictors (correlation coefficients ranging from .6 to .8) which seem to fall in two seperate clusters. I am using stepAIC (in R) and LASSO for model selection, and one particular predictor (which is highly correlated with another simliar predictor) always ends up in the final models with a positive contribution, although it being negative would clearly be the most intuitive. The second predictor however, always shows a negative contribution. Could this be caused by the high correlation between these two predictors?
Also, can you include non numerical predictors (say Yes/No values) in a correlation-diagram by simply assigning numerical values to them, like 0 and 1?
3. Diagnostics for logistic regression
I have come across a lot of different suggestions of how to perform diagnostics for logistic regression models. To me, one of the mroe appealing approaches was residuals plots.
The plots below show the predictors (all continuous) plotted against the residuals of a final model chosen through stepAIC. My initial thought is that there are no clear patterns in the residuals that would suggest adding more complexity to the model (through interaction terms or variable transformations). Are you sharing this point of view, or am I missing something?
The deviance test for assessing goodness of fit for the model (call it m) is carried out in R as follows (correct me if I am wrong):
1 - pchisq(deviance(m), df.residual(m)).
For my particular model, i get a p-value of .87. This should be telling me how plausible it is for the model to have generated the data, but is it assessing the linear assumption of the log odds given in the logistic regression model? Are there other ways to assess the linearity assumption?
4. 'Pseudo' R-squared
Is it common practice to report a 'pseudo' R-squared for predictive performance, when implementing logistic regression? The McFadden and Cox & Snell to me seems the most recommended ones, should you choose to report one.
Thanks in advance!