I have been developing a propensity score matching model, using logistic regression to model propensity, and I am wondering about recomputing propensity scores in order to validate a model. Here is my hypothesis, please tell me where I am wrong: If propensity score matching "works" and we have a control set that is balanced on the treatment set, if we recompute the propensity scores and plot them both, they should be normally distributed with mean 0.5.
I took some data I was using, and constructed a control set in the following way: found the mean and variance of each covariate in the trial set and then made a control set that randomly samples from the normal distribution but with treatment = 0. When I did that, sure enough, the propensity scores were distributed around mean 0.5.
This makes sense to me because if you have adjusted for covariate balance, then it is a coin flip whether they received treatment or not, right? And using logistic regression, there is basically no signal as to whether a person is treated or not, so p ~ 0.5.
If so, why wouldn't a diagnostic of whether propensity score matching worked be to recompute the propensity scores for the constructed control group and see if they cluster around mu = 0.5?