Why the predicted probalility in logistic regression is small, spreading within (0.0, 0.6), rather than(0.0, 1.0) Just realized that the predicted probability of default in logistic regression is small and spread within a narrow range. My first guess is that it could be related to the not-strong-features selected during the modeling. My question, is this normal or did somebody observe this phenomena before? how to correctly transform as it represents the 'True Probability" spreading within (0.0, 1.0)
thanks.
 A: If your features are not overly predictive, then your predicted probabilities won't be very high.
For instance, if you are classifying people as having a particular disease or not, and all you have is their temperature, the predicted probability that someones suffers from this particular disease will be small - after all, a high temperature could be due to any number of other diseases.
Work on getting better predictive features - in the disease example, blood tests, a look down the patient's throat etc. It doesn't make sense to transform the predicted probabilities from a logistic regression that does not cleanly separate the classes.
Alternatively, your final classification can use a different threshold than 0.5. Nothing is stopping you from classifying samples with a predicted probability of 0.3 as being in the target group. What probability threshold you use should be governed by the relative costs of Type I and Type II errors.
A: Maybe is the range of the explanatory variables:
If you have temperatures (according to other users' example) vs probability of having the disease, the probability at the maximum measured temperature is 60%.
Logistic regression model doesn't know you are talking about temperature (is just another continuous variable). So the model could be saying that "The probability of having the disease is 95% when the temperature is 95°C (really high)". The model doesn't know that this temperature would have killed a human way before has been reached.
So, in conclusion: Having (0.0,0.6) is not an incorrect output, it reflects the reality (but with flaws)
