calibrate function in the
R package allows us to compare the probability values predicted by a logistic regression model to the true probability values. This is easy enough: just plot them and make sure they are about the line $y=x$. However, if we knew the true probability values, there would not be any need to do statistical inference! We have to estimate what the true values are, which is a major part of what
rms::calibrate does and what confuses me.
I have figured out a bit of how
It estimates the true probability values using the observed $0/1$ labels. This makes sense. If we have a lot of $1$s, we should have a lot of high values of $P(Y=1)$.
It takes bootstrap samples of the predicted probability values paired with the observed outcomes.
It fits some kind of model with the bootstrap samples of predicted probability values paired with observed outcomes.
Step $3$ is where I start to get lost. What kind of model is fit? Is it, indeed, fit to the bootstrap samples of predicted probability values and observed outomes, or does it fit a new logistic regression each time? (I don't think this is how it work, but I want to make sure.) How does it go from multiple models of bootstrap data to the "apparent" and "bias-corrected" values displayed on the graph?
library(rms) set.seed(2022) N <- 1000 x <- runif(N, -3, 3) z <- x pr <- 1/(1 + exp(-z)) y <- rbinom(N, 1, pr) plot(x, y) L <- rms::lrm(y ~ x, x = T, y = T) cal <- rms::calibrate(L) plot(cal)