Multicollinearity in Text mining regression analysis (NLP)

Question

I am trying to predict CTR (which bigrams increase/decrease CTR) on a document term matrix (built on bigrams and term frequency weighing) and building a generalized linear model where I have transformed the dependent variable to logit function (log(CTR/(1 - CTR))) since initial CTR is a bounded function 0 to 1). Then, building a regular linear OLS model

Text variables majorly have values 0,1 or 2 and DTM is sligtly sparse.

After building the model, I am seeing some non-normality & hetroscedasticity in my residual plots. Below are some questions:

1. Is use of logit function correct here or some advanced technique?
2. I checked vif here and found a lot of multicollinearity among text variables. Do we remove multicollinearity in text variables while running regression or we can safely assume independence and go ahead with OLS regression? If yes, how to remove it in case of text variables?
3. How to I fix for non-normality or hetroscedasticity in my model?

Thanks in advance.

Answer 1

1:

Logit function is not well suited for such outputs. Poisson regression comes to mind for count data.

2:

Did you try to add regularization to your model? It can help with multicolinearity.

It seems like good idea to try lasso/elasticnet in particular, since they are suited to sparse data.