In my dataset, all the variables are highly correlated (correlation coefficient > 0.95). However, the correlation with the dependent variables is very low (<0.35). I checked the variation inflation factor (VIF), but it's 'inf' for some variables and very high (around 6 figures) for other variables. So cannot filter with the threshold of 10 as usual but setting a threshold around 1000 does not seem to be correct either. The response is a binary outcome. I'm trying to build a logistic regression model but due to high collinearity coefficients are not interpretable. How can I handle multi-collinearity in this case?
EDIT: I want to use the model to check what variables are significantly associated with the outcome and also to predict the outcome. The training set size is 128 (after splitting into train-test) and there are 96 variables in total. Data is highly imbalanced.