# Probability prediction using logistic regression for an unbalanced dataset [duplicate]

I am trying to estimate the probability for a user to get to a specific page in a website. The collected information for the users are

• countries
• whether they use a PC or a mobile device
• referral page

All the variables are categorical and I have two possible outcome, either success (1) or not success (0).

I thought of using logistic regression as my dependent variable is categorical. I would get the probabilities for the two classes using the predict_proba method in sklearn implementation of the classifier. However, I have some doubts:

1. The classes are not perfectly balanced, i.e. $\sim$1 in 10 users has outcome 1. I compared my model with a dummy classifier and they equally perform;
2. performing a grid search over the classifier, any combination of values leads to recall of zero. That is not only does the classifier outperform the dummy one, but does it not find the success-labeled users.

Given an intelligent threshold as said in this similar question, could I use the prediction to assess such a success probability for a user? And, given that I have 3 categorical variables, should I think of the observations/variables ratio before or after the one-hot encoding process?

• Generally speaking, and according to [1], using an imbalance dataset may have an effect on the intercept coefficient value and, consequently, the estimated probability. [1] King, G. and Zeng, L. (2001). Logistic regression in rare events data. Political Analysis, 9(2):137–163. – PatternRecognition Jan 5 '18 at 14:56
• A word on informal terminology. Highly imbalanced usually means something like 0.01% of one class, i.e. severe rarity of an event. 10% prevalence is well within the domain where no special methods are needed to account for the class rarity. The ML community has a weird obsession with perfectly balanced datasets, but it's worth casting a skeptical eye towards. Good on you for going with predict_proba. – Matthew Drury Jan 8 '18 at 15:22
• @MatthewDrury thanks for the explanation, I edited the question and removed the tag. – Mattia Paterna Jan 8 '18 at 15:32

• Thanks for your answer Stephan, I really appreciate the related links. The point here is that, given my model, I will never get a probability of success $p > 0.5$ - and I agree with it. Therefore I was interested in directly giving such the probabilistic prediction but I did not know if it was completely correct. – Mattia Paterna Jan 8 '18 at 8:22