1
$\begingroup$

I'm trying to understand how to interpret changes in the predicted probability of an individual being classified (by logistic regression).

This feels like either something that is fundamentally obvious, or so entirely nuanced I'm naive in thinking it has a general answer.

Let's say my first model produces a range of probabilities that individuals are classified A vs not A. Some time later, I run a similar model against the same individuals, but with updated dependent variables/outcomes, and, accordingly, the probabilities change. A particular segment 'S1' might have a probability of say 0.7 in the first model, and it in fact goes to 0.8 in the second. I understand that this can mean that of 100 of these, the net outcome will be 10 more being classified as A; and this is borne out by the underlying data. In segment S2, the probability goes from 0.45 to 0.55 - again of 100, 10 more will be classified as A.

So far so simple, I thought, but I get stuck at trying to interpret what happens at the individual level. There is a strong intuition that S2 are more likely to have changed from ~A to A, arising from the probability changing from 0.45 (~A dominant) to 0.55 (A dominant). But in both segments, 10% have changed. Which rejects the intuition...

Can anyone help get me out of my confusion? I think I need to look at underlying distributions, but not quite sure.

Improve this question
$\endgroup$
7
  • $\begingroup$ Are you fitting different models on the same train set? You should expect that as you build a more complex model (on a fixed data set), it increases/decreases posterior probabilities (predictions) in certain subspaces; however, the average posterior prediction should remain fixed (equal to the train set class priors), provided you've included an intercept in your model. Details: stats.stackexchange.com/a/16541/187294 $\endgroup$
    – khol
    Commented Apr 24, 2018 at 17:44
  • $\begingroup$ Thamks for your response ... Same model formula, on same individuals....but the second run has more data, hence some of the probabilities changing. $\endgroup$
    – N Mason
    Commented Apr 24, 2018 at 18:53
  • $\begingroup$ "Same model formula, on same individuals" conflicts with "second run has more data". Re-reading your question, it sounds like your data set (both predictors and response) are being updated between fits. Unless you can specify how these are changing, there's nothing surprising about predicted posteriors changing magnitude. $\endgroup$
    – khol
    Commented Apr 24, 2018 at 19:18
  • $\begingroup$ I would also suggest being careful with how you are interpreting the results of the regression. A logistic regression does not "classify" anything; you may use the posteriors it produces to make classifications (i.e. by picking a threshold), but the predictions it produces are much richer than just that. $\endgroup$
    – khol
    Commented Apr 24, 2018 at 19:23
  • $\begingroup$ Many thanks for the responses; helpful, but probably indicating that I'm not really describing my question perfectly. But you're helping me think through what is the nub of the question Essentially the DV - a binary outcome - can change for each individual in the study, between the first set of data and the second. What I'm keen to understand is whether the actual change at an individual level is deducible from the model - and specifically whether the likelihood of a change in the outcome is independent of the modelled probability. $\endgroup$
    – N Mason
    Commented Apr 24, 2018 at 21:41

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.