I created a prediction model using a multiple logistic regression on a non-representative sample of the target population.
With this model, I can get a probability for a new patient of belonging to class 0 (No Disease) or 1 (Disease) of my outcome variable. The probability in the target population of belonging to class 1 (i.e. the prevalence) is 10%, vs 40% in my sample.
Let's consider the 2 following situations
- The patient belongs to the target population (Disease prevalence : 10%)
- The patient belongs to a population where the disease prevalence equals the disease prevalence in the sample used for the model development (40%)
Questions
If I apply the model and get a 50% predicted probability of Disease, would the true probability be identical in these 2 situations (50%)? In other words, do have to take into account the disease prevalence to get the probability that my patient has the disease? And if so, how?
Or do I need to go through a sensitivity/specificity calculation and then (positive/negative) predictive values and therefore use a decision threshold based on the predicted probability?