# Logistic regression model; too many independent variables?

I'm building a logistical regression model to predict the gender of users based on listening duration of music genres. My main worry is that I have over 40 different genre's and I'm concerned that having that many independent variables is going to hurt my GLM, or make it uninterpretable.

Is there a better way to approach the problem?

Thanks!

• You maybe risk a lot of multicolinearity. There is also the risk of quasiseperation. I think if prediction is your main goal, adding a penalty term should take care of both of these worries. – Demetri Pananos Nov 7 '19 at 16:22
• @pr1g11 I would also caution you that by including that many variables will make you capitalize on the chance that a couple of your variables will be statistically significant by chance alone. – PsychometStats Nov 7 '19 at 16:22
• Lasso and ridge should work, but if you use those you can't perform inference. What is most important to you right now: inference of prediction? – Demetri Pananos Nov 7 '19 at 18:01
• My main aim is to build a predictive model so maybe this is the best way to tackle the problem. My current game plan is to split my data into training and test sets (using cross validation techniques), run the LASSO regularisation and then perform the GLM on the variables which the Lasso doesn't reduce to zero. Without seeing my data, does that sound like a reliable methodology? Thanks for the help btw @DemetriPananos – pr1g11 Nov 8 '19 at 16:40
• Hmm, I don't think that is the best way to go about things. You should a) Split into training and test. Don't touch the test set. b) use cross validation to select the best regularization strength. Don't just select the non 0 parameters, you can still get a good fit when variables are left in the model. c) Once you've selected the best regularization strength via CV, estimate your out of sample error using the test set. Create a confidence interval for the loss. – Demetri Pananos Nov 8 '19 at 16:44

If a categorical input has too many levels, in your case too many genres, you can use a statistic trick collapsing categories by thresholding. For example you can use the zip codes to map to several relevant demographic variables such as median home value, however, the levels of zip codes are too many.

Collapsing categories by thresholding method require a minimum number of cases in a level in order to create a dummy code input for that level. For example, you can do the count of each genre, if the count is below a certain value, you can collapse with another genre and give a "new name".

Another way is to collapse genres by background knowledge, if you think pop and hip-pop are similar, you can just collapse them in one genre.

Hope this helps. Reference is from the statistical learning book.

You can use penalized logistic regression.

To learn more, see these threads: What is penalized logistic regression and Penalized Log-Likelihood - Logistic regression.

• Just just run the model with all the independent variables, and 5 have come out as significant. I could run a stepwise backward regression, and only keep the variables which are significant. These are also the variables which correlate to the target variable. Is this bad practice? – pr1g11 Nov 8 '19 at 16:35