Logistic regression: How to improve the pseudo-$R^2?$ Firstly, I would like to ask you opinion about my model: I have a logistic regression model where the dependent variable is late payments (1=late/0=not late) and diesel prices and interest rates as the independent variables. I have checked for the VIF factor which less than 2 for each feature. These give p-values 0.000.
Secondly, would anyone know how could I improve the model? The pseudo $R^2$ value is just $0.02511.$ My goal is to find explanatory variables that could explain the late payments, and I tried several others, but they resulted with high VIF scores.
 A: VIF should play no direct role.  What are the two frequencies of the outcome variable?  Why late vs not late as opposed to number of days late (with right censoring)?  You might do some variable clustering (unsupervised learning) to group some collinear variables but don't exclude variables because of collinearity.
See my notes for some useful information about pseudo $R^2$ and adjusted versions of same: https://hbiostat.org/rmsc/lrm.html#quantifying-predictive-ability and the link contained there.
If you have done due diligence in using subject matter thinking to specify a model, and possibly done some unsupervised learning to live within the confines of your effective sample size, a low $R^2$ may be telling you that it's just hard to predict that outcome.
A: In order to increase the performance of a given model we need to provide more information to our modelling procedure. That is done almost exclusive in two main ways: 1. we increase our sample size by adding new entries (either artificially by transformations or naturally by sampling additional points) so we know "more" about the underlying population and 2. by increasing the expressiveness/available features of our existing sample such that we know  "more" about the existing sample. The former point is what we call "data augmentation" and the latter point is what we call "feature engineering". So if in this case the model is fixed to be a logistic regression model, we have to explore how these two options are applicable to our problem. While harsh, it has to be one of the two. Or we change the model entirely (e.g. a random forest) but that's another game. For example, what Stefan says, about the "use domain knowledge to improve your model" is effectively "expert feature engineering", while if you can get more samples is "real dataset augmentation".
As Frank mentioned, the VIF has little to do with the observed goodness-of-fit of a model. Multicollinearities are not a big problem in terms of predictive performance but more in terms of increased instability of predictor selection; see Performance of binary prediction models in high-correlation low-dimensional settings: a comparison of methods (2022) by Leeuwenberg et al. for more details.
If within a regression framework, we are worried about multicollinearity and multiple variables being included, we can look into an elastic net logistic regression model at first instance. While not a panacea, the elastic net penalty will allow some degree of regularisation both for collinearities and multiple variables being included. Standard calculation of $R^2$ on the likelihood scale (McFadden, Cox and Snell, and Nagelkerke) work as before. As mentioned already though expectedly these will be even more unreliable than their unregularised counterparts in small samples; see Penalization and shrinkage methods produced unreliable clinical prediction models especially when sample size was small (2020) by Ripley et al. for more details.)
