I have an existing logistic regression used to forecast whether a given action will take place or not. The unit of analysis is the individual person. For the sake of the question, assume a default threshold of
p > .5 yields a positive prediction. When I plot the proportion of individuals who are actually in each class for a given day vs. the proportion of individuals predicted to be in each class for a given day, the model drastically underpredicts. For example, the model will predict that 8% of individuals will be in class 1 whereas the actual proportion is 13%.
However, it was suggested that I take the average of the predicted probabilities for all individuals on a given day to predict the proportion of people in each class. This actually works very well (within a % or so).
How is it the case that a model that is individually quite poor is collectively quite good? Averaging predicted probabilities does not seem like a sound solution to me, but I cannot figure out quite why.